The MCCCC COE Impact Model

In developing the COE Impact Model (COE-IM), we adopted a business approach of a functional model that clearly connects the CCSG COE criteria to the MCCCC goals through COE short-term and long-term outcomes (see Fig. 1). The COE-IM is anchored by the CCSG COE requirements [1] and the MCCCC goals for its CA communities. The CCSG requirements [1] for COE include:

Define and justify the center’s CA.

Identify the major factors that characterize and influence the cancer burden in the CA.

Describe how the center has reached out to and engaged with community members and organizations in its CA to inform cancer research and control efforts of relevance to the CAs.

Describe how the center has communicated community needs to center members and catalyzed research in all areas of science.

Describe the role the COE component has in facilitating accrual to clinical trials from the center’s CA.

Describe the cancer control efforts undertaken by the center to reduce the burden of cancer in its CA.

Describe how the center has expanded its reach, including globally.

Fig. 1

Mayo Clinic Comprehensive Cancer Center Community Outreach & Engagement (MCCCC COE) Impact Model to describe specific processes and outcomes to meet overall MCCCC goals

With a vision for a cancer center of the future, the mission of the MCCCC is a center without walls building upon all Mayo Clinic strengths, delivering cancer expertise to the patients and communities served by MCCCC anywhere, anytime and for anyone. MCCCC strives towards this mission by connecting patients and communities with data, intercepting pre-cancer, developing more effective treatments, transforming cancer care delivery and transforming cancer clinical trials to achieve six population health goals: (1) decreasing cancer morbidity in MCCCC CAs, (2) decreasing cancer mortality in MCCCC CAs, (3) decreasing cancer health disparities in MCCCC CAs, (4) increasing the representation of minoritized and marginalized communities in clinical trials and biomedical research, including tissue donation, (5) increasing the representation of underrepresented minorities (URMs) in biomedical research workforce, and (6) increasing global oncology research capacity. In alignment with the MCCCC goals, the long-term objectives of the COE are to increase:

Population knowledge/awareness of cancer, cancer risks, cancer prevention and cancer screening.

The uptake of primary and secondary cancer prevention and risk reduction behaviors /strategies.

Participation of target populations in clinical trials and biomedical research.

Cancer-related health equity research and extramural funding.

To complete the COE-IM, short-term objectives that will facilitate the achievement of COE long-term objectives were developed in alignment with the CCSG COE criteria. For the CCSG criteria focused on CA definition, CA characterization, engagement of CA communities on cancer research and cancer control efforts, and engagement of Center members on community needs, the short-term objectives are to increase: (1) center staff and researcher awareness/knowledge of the CA priorities and needs, (2) research and extramurally-supported awards on CA populations, (3) educational programs offered at CAs, (4) representation of CA populations on research studies, and (5) community awareness of cancer needs. For the CCSG criteria focused on the engagement of center members on community needs, and the center’s cancer control efforts, the short-term objectives are to increase: (1) the number of minoritized and marginalized communities reached/educated relative to cancer prevention and risk reduction, (2) cues to action, including people and things that activate behavioral change, (3) knowledge of participants relative to cancer prevention and control, (4) cancer awareness, (5) positive attitude relative to cancer prevention and risk reduction, and (6) behavioral intention relative to cancer prevention and risk reduction.

For the CCSG criteria focused on facilitating accrual to clinical trials, the short-term objectives are to increase: (1) the number of minoritized and marginalized communities reached/educated relative to clinical trials, (2) cues to action that activate accrual to clinical trials, (3) clinical trials knowledge of participants, (4) clinical trials awareness, (5) positive attitude about participating in research/clinical trials, (6) behavioral intention relative to participating in research, (7) positive attitude about donating samples for research, and (8) behavioral intention relative to donating samples for research. For the CCSG criteria focused on expanding the center’s reach, the short-term objectives are to increase: (1) research collaborations with investigators from Low- and Middle-Income Countries (LMICs), (2) global oncology research and extramurally supported awards; (3) global oncology research training in LMICs, and (4) global oncology research capacity in LMICs. The COE-IM includes a strong program evaluation to evaluate the impact of MCCCC COE objectives for continuous quality improvement.

The MCCCC COE Logic Model: foundation for continuous improvementRole of program evaluation in facilitating the operations of COE

In order to gain an in-depth understanding of the impact of COE initiatives, program evaluation frameworks in public health are often used to document and monitor the progress of activities. From a utilization-focused evaluation approach, program evaluation is valuable to facilitate the operations of COE by 1) identifying a program’s goals and priorities 2) improving overall program effectiveness, and 3) guiding future decision-making [4].

Identify priorities

A formative evaluation can be useful in the initial states of program development, in determining specific needs, establishing processes and plan data collection in preparation for implementation [5]. For example, as a MCCCC COE initiative, the cancer-focused Community Needs Assessment (CNA) was developed by an interdisciplinary team of scientists to understand CA needs and priorities. As results of the cancer-focused CNA are reported, a feedback approach for responding to community’s needs will be developed through program evaluation. The cancer-focused CNA incorporates population and sociocultural constructs to account for diverse groups in CAs and will be instrumental in shaping programs to reduce cancer burden in marginalized groups.

Improve effectiveness

Summative evaluation helps to determine impact of a program, if the goals and objectives of the program are being met, and if the program accomplished what was originally intended [6]. To improve effectiveness, outcomes are assessed, and strategic decisions are made to modify program components to better meet the needs of the community. As an approach to enhance current efforts and maximize impact in COE, a training curriculum on COE evidence-based intervention was initiated, based on program planning frameworks. First, an inventory of current COE programs was assessed and categorized with preliminary evaluation outcomes. Second, programs with highest impact and strongest partnerships were prioritized for re-implementation. Third, management staff underwent training for evidence-based planning, implementation, and evaluation, and developed operational plans for evidence-based program implementation in their subject areas of expertise. Focused training on evidence-based interventions and practices in promoting cancer health equity has strong potential to improve effectiveness of programs by integrating scientific rigor in COE efforts, developing evaluative measures to document progress and demonstrate accountability, and building capacity for established and active community partners invested in reducing cancer health disparities.

Guide decision-making

Following implementation, the results of the evaluation can include a) summary of overall outcomes relative to program goals and objectives, b) highlights of accomplishments, c) progress towards goals of specific programs, and d) recommendations for improvement that are communicated to both internal and external audiences and stakeholders for ongoing program improvement. This aspect of program evaluation is particularly useful to facilitate evidence-based decision-making. To foster collaborative relationships, community feedback is interwoven in planning for program improvement and improves agency of local partners in COE efforts. In COE operations, program evaluation is vital in gaining understanding of strategies for sustainable, scalable programs to reduce unnecessary costs and efforts, improving community health disparities, monitoring overall progress, and maximizing impact of collaborative evidence-based programs. With an effective evaluation plan, outcomes based on predefined goals can be carefully monitored and tracked, yielding valuable insight on achievements and gaps on progress.

Use of Logic Model in operationalizing Impact Model

Logic models are used in tandem with evaluation plans, often as a blueprint for programs’ resources, activities, and expected outcomes. A logic model can outline what a program is trying to accomplish, how to pinpoint specific areas of assessment that is most needed, and at which timepoints evaluation tools should be disseminated [7]. Driven by theories of change, a logic model is used to identify clear objectives and measure expected results of a program or initiative to improve impact and value [8]. To operationalize the aims defined in the COE-IM, a Logic Model was developed in line with the MCCCC COE goals for CA communities. The logic model was developed in line with the CCSG guidelines, driven by MCCCC long-term goals, which reflect significant efforts to decrease cancer disparities in risk factors, incidence, prevalence, morbidity and mortality, and increase access to appropriate cancer education, resources, and care particularly for minoritized and marginalized populations. The COE Logic Model (see Fig. 2) visually depicts the sequence of progress towards long-term impact and overall goals for MCCCC COE initiatives.

Fig. 2

Mayo Clinic Comprehensive Cancer Center Community Outreach & Engagement (MCCCC COE) Logic Model developed to identify objectives, and track and monitor activities for process improvement of COE activities

The inputs include items invested (time, money, staff equipment, etc.) and needed to generate outputs focusing on the reach of specific populations, partners, and groups, through various activities, to develop tangible products which are instrumental in operations towards achieving goals. The short-term goals describe the intended changes in knowledge and learning, while the intermediate outcomes are expected over a longer period and focus on the utilization or changes in action. The long-term outcomes are aligned overall goals of MCCCC COE and outline the societal impact of COE programs in the CA communities.

MCCCC COE inputs

Identifying the resources available and investments required to operationalize the COE-IM represents the foundation of the COE Logic Model. The inputs, what MCCCC COE invests in both quantity and quality, determine the success and sustainability of the programs. In addition, this foundational process reveals gaps in resources and the barriers or constraints that require identifying critical strategic factors to overcome [9]. For the COE Logic Model, the inputs are: (1) money; (2) time; (3) personnel; (4) partners; (5) expertise & knowledge; (6) materials; (7) equipment; (8) space; (9) technology; (10) community research partners; and (11) community input and involvement. The following descriptions of each input are provided.

Money

MCCCC COE programs, as a part of an NCI-designated comprehensive cancer center, are funded from the NCI CCSG award, non-CCSG grant awards, institutional and philanthropic funds. In addition, COE has been identified as a strategic priority within MCCCC for potential benefactors by its development team.

Time

The COE Logic Model reflects three different considerations for time: time needed to build community trust; sequencing of inputs and activities to produce desired outputs; and timelines/milestones that measure progress towards identified strategic aims. The COE transition team established two-year and five-year objectives, which served as the initial filter for inputs required in the short term versus long term.

Personnel

A key strength of the MCCCC COE personnel is the diversity of background, thoughts and worldview that represents our CA populations. The COE Office and Programs is directed by an enterprise-wide Deputy Director, with Associate Directors designated for each of the three sites in Arizona, Florida and the Midwest. There is also COE Faculty members, comprising 9 faculty with expertise in community engagement research. The COE Operations team is led by an Operations Administrator and a COE Director at the enterprise-level. Each site has a project manager, outreach coordinators and trainees.

Community input and involvement

According to the NCI CCSG guidelines, Community Advisory Boards (CABs) are crucial to ensure bi-directional communication as well as receive consistent and timely feedback from the community [10]. MCCCC COE has a CAB comprised of a diverse group of community leaders, cancer survivors, and advocates representative of its CA. The MCCCC COE CAB serves as a connector between the MCCCC and the CA communities served by the MCCCC through community engagement, partnerships, and outreach. Representing the rich diversity of the MCCCC CA populations, CAB members are thought leaders that advise, prioritize, and participate in the development of solutions that include community-engaged research, cancer prevention and control activities, and clinical care efforts to address the cancer needs of our diverse catchment area population. In addition to the CAB, community members have the opportunity to be involved with researchers through the MCCC COE’s Community Engagement (CE) Studios [11] and Citizen Scientist program.

Partners

Partnership is a key input of the MCCCC COE. The COE Office and Programs strategically forms partnerships that are long term and bi-directional, including formal and informal coalitions and collaborations. The MCCCC COE has over 200 partners that range from community-based organizations, faith-based organizations to community health systems that serve minoritized and marginalized communities, including federally qualified health centers.

Expertise

Apart from academic and professional knowledge in relevant areas, passion for social justice is a common thread across both external and internal stakeholders. Additional skills and abilities held by stakeholders include emotional intelligence, organizational, management, interpersonal and communications, which are all skills vital to engaging and meeting the needs of the MCCCC CA communities.

Equipment and materials

The MCCCC COE Office and Programs provides shared resources for the community and MCCCC members nationally and globally. As an example, MCCCC has provided digital signages, computers and large screen monitors at the community level to develop COE resource center hubs in several community sites. Equipment that has been provided include freezers and ultrasound equipment for biobanking at international sites, as well as solar panels for electricity in low resource settings.

Space

The nature of community outreach and engagement work requires reaching the CA populations where they are. It is thus imperative that MCCCC COE has a consistent presence where individuals live, work, and reside. In Florida, Mayo Clinic established the Mayo Clinic Community Health Collaborative (MCCHC) within the Jacksonville community as a community-based resource center to foster outreach, education and community engagement research, including minimal-risk clinical trials. Across the enterprise, we have partnered with multiple organizations with physical presence in minoritized and marginalized communities using a hub and spoke model. This includes the American Legion Post, Urban League, YMCA and County extension offices.

Technology

The use of technology to reach minoritized and marginalized communities is a critical tool to address cancer health disparities. However, this must be done in such a way that it does not create a digital divide for the CA communities. MCCCC COE input in this area includes setting up digital health platforms and video communication platforms at community COE spokes. In addition, MCCCC COE is currently working with the Mayo Clinic Platform team to develop and make public the profiles of CA counties, including population demographics, socio-demographic factors, behavioral risk factors, cancer screening, and cancer burden and cancer patterns. Preliminary profiles have been successfully used to support COE mini grant applications by community-based organizations. All applicants were provided the profiles for the CA counties targeted for their applications, making it easier for the applicants to target the needs of their communities.

MCCCC COE reach

Community stakeholders impacted by MCCCC COE ranges vastly across both the internal and external environment. Examples of the COE reach, or who the Center touches are described below.

MCCCC programs and members

MCCCC COE supports MCCCC members for community engaged, community-based participatory, health equity, health disparities, and/or population health research through multiple activities, including monthly help sessions and tailored consultative services. Additionally, the COE Office and Programs works closely and synergistically with other MCCCC Offices and Programs. Figure 3 provides a summary of these partnership activities.

Fig. 3

Various activities conducted by Community Outreach & Engagement (COE) which provides support for Mayo Clinic Comprehensive Cancer Center (MCCCC) members for community-engaged, community based participatory, health equity, health disparities, and/or population health research

Community partners

MCCCC impacts individuals, families, groups, community organizations, and private institutions within its CA, spanning 75 counties across five states. MCCCC COE aligns its interventions and activities across the cancer continuum, from prevention to survivorship and advocacy, to maximize its impact and engage these partners in the research process. MCCCC COE partners include clinical partners, faith-based organizations, and community-based organizations.

Community members (public)

MCCCC is well represented by its CA communities through the COE administrators, faculty, staff and CAB members. In addition, the COE conducts a cancer focused needs assessment to identify the CA’s needs and priorities that augments the federally required Community Health Needs Assessment (CHNA). These community inputs allow MCCCC COE to address identified issues, that touches the public, to improve community health and reduce the cancer burden. For example, in Florida, MCCCC COE partnered with the American Legion Post #197 and the Jacksonville Urban League to establish community infrastructure supportive of community research and initiatives.

Providers

The strength of Mayo Clinic lies in its ability to address serious and complex health conditions. One of MCCCC COE’s priorities is thus reaching the health professionals and organizations that serve CA communities with opportunities to participate in critical research that contributes to possible health cure. As an example, MCCCC COE partnered with The Way Clinic, Barnabas, Aza Health, and Flagler Free Clinic to increase outreach and research activities in rural Florida.

Patients/survivors

The COE activities cover the cancer care continuum, including outreach for survivorship care that are culturally and ethnically appropriate, and inclusive research for patients and survivors.

Trainees

MCCCC COE supports underrepresented students, interns, postdoctoral fellows, and early-stage investigators with opportunities to learn about and be involved in community-based research. In collaboration with MCCCC Cancer Research Training and Education Coordination (CRTEC) and Diversity, Equity and Inclusion (DEI), the Office also organizes webinars and workshops focused on community engagement research and community outreach initiatives.

Academic and medical partners

Collaboration with other academic and medical institutions enables MCCCC COE to significantly impact the cancer burden within its CA. Universities, research institutions, and Federally Qualified Health Centers as partners in research studies and clinical trials acts as a force multiplier to address public health concerns that affect everyone. Examples of MCCCC COE academic and medical partners are Adelante Healthcare (AZ), Community Health Services, Inc. (MN), and Agape Health (FL).

MCCCC COE activities

The MCCCC COE activities to reach populations, partners, and groups in the CAs, include activites developed through an iterative process that includes internal and external stakeholders. Examples of these activities are provided below.

Creation of external community-facing data warehouses, platforms & visualization tools to guide interventions /trials

The community facing platform provides community members and external stakeholders with information about the cancer burden that impacts the 75 counties across the MCCCC CAs. The community cancer profile by county includes county demographics, cancer incidence and mortality and cancer screening rates. MCCCC COE is working with CA communities to develop and implement activities that address the cancer burden in the CA counties.

Identifification of CA cancer priorities and needs to support MCCCC investigators, patients & CA communities

The Cancer Needs Assessment is currently being dissemenated throughout the MCCCC CAs. There are several measures, which includes, demographics, cancer family history, lifestyle behaviors, and social determinants of health. The assessment utilizes several methods of data collection, including push to web surveys, survey panels, as well as, in-person data collection.

Established MCCCC-funded community mini-grants

The primary goal of the MCCCC COE Community Mini-Grant program is to facilitate the elimination of cancer health disparities by reducing cancer burden in the communities served by Mayo Clinic. The Mini-Grant awards up to $20,000 to community organizations for cancer focused projects. Successful awardees work closely with the MCCCC COE Office to implement evidence-based intervention programs to improve health at the individual, community and/or health system level.

Established Native American Health Program

The Native American Health Program includes culturally tailored activities aimed at improving the cancer burden in the Native American communities served by MCCCC COE. Specific activities include tribal focused cancer needs assessment and partnership building with community organizations that serve Native Americans.

Community forums and summits

MCCCC COE regularly hosts culturally tailored community forums and summits focused on cancer-related topics across the cancer care continuum. This includes the Cancer Conversations series that features medical experts and survivors providing information about different cancers. The annual Mayo Clinic Partnership Summit brings together MCCCC members and community members to foster relationship building, engagement, and bi-directional communication for engaging community stakeholders.

Partnerships with community providers to facilitate referrals for clinical trials

MCCCC COE continues to build partnerships with community clinical providers, including those in private practice, community health centers, as well as Federally Qualified Health Centers. The goal of the partnerships is to understand the specific cancer needs of their patients, the research interests of the community providers, and the capacity of providers to partner on the development of a decentralized clinical trials model that will facilitate increased community access to MCCCC clinical trials. One example of this is MCCCC COE’s partnership with the Northeast Florida Clinical Trials Consortium, which is made up of Black physicians serving patients in Northeast Florida.

Description of MCCCC COE products

Examples of the COE Office and Programs products are summarized below.

Internal-facing & external-facing data platforms

The interactive platforms provide internal and external stakeholders with a variety of data, including CA demographics and cancer burden by county (incidence/mortality rates, cancer risk behaviors, screening rates, etc.), internal and external resources to address social determinants of health.

Catchment area cancer priorities and needs report

The MCCCC COE Community Cancer Focused Needs Assesment survey is currently ongoing with data collection in CA counties. The results of the survey will be analyzed to determine the community level needs across the MCCCC CA. The analysis will be developed into an internal report that will be disseminated to MCCCC members to faciliate clinical and research activities, as well as, a community report that will be disseminated externally to facilitate the development of activities to address the identified needs and priorities.

Decentralized Clinical Trials infrastructure

In an effort to faciliate increased community access to clinical trials, the Decentralized Clinical Trials infrastructure is a “hub and spoke” model. In this model, MCCCC sites serve as the hub of clinical trial implementation (i.e. development of trials, regulatory activites, primary study coordination, etc.). The spokes of the model include resources within the CA to improve access. Examples of the spokes resources are:

Mobile Research Unit (MRU): The MRU is an RV-style vehicle that is a self-contained, mobile research facility. It includes two exam rooms, equipment for lab tests, a private area for participant interviews, audio/visual technology for participant education, and a laboratory. The Northeast Florida based unit is used to bring minimal risk research studies to rural and hard-to-reach communities thereby expanding access to research studies.

Community Based Sites: The community sites serve as locations for varying clinical trials, including, but not limited to, recruitment, research interventions, and trial follow-up activities. The sites range from community practices, Federally Qualified Health Centers, Community Health Centers, and other community based organization locations.

COE multimedia materials

MCCCC COE disseminates information to internal and external stakeholders addressing cancer across the continuum, foster bi-directional communication between community stakeholders and MCCCC members to faciliate research participation, and provide updates on COE activities and results. Materials are disseminated through community reports, websites, social media, print materials (newsletters, infographics, etc.), and video format.

Community-based training programs

In an effort to faciliate research participation in MCCCC research activites, COE has developed several training programs for MCCCC members and communities. For example, COE developed a COE Faculty training program focused on Community-Based Participatory Research (CBPR), cultural humility and responsiveness to successfully engage, recruit and retain diverse research participants. COE also developed a Citizen Scientist Program and Community Health Worker (CHW) Cancer Training Program to educate and empower community members to engage in the research process and understand key cancer topics across the cancer contiuum.

MCCCC COE short-term and intermediate outcomes

When examining the short-term outcomes and impact of COE efforts, the measurements involve the extent of change primarily in knowledge and learning. Expanding the reach of COE can ultimately impact awareness, utilization, and representation of cancer-focused resources, particularly in the CAs. The impact of outcomes in the short-term include increase in individual-level factors such as knowledge, attitudes, and satisfaction regarding general understanding of cancer disparities and available resources. Additionally, the quality, use, and perceived usefulness of COE materials are expected to increase. Finally, short-term impact can also include an increase in awareness, behavioral intent, perceived behavioral control, motivation, and informed decision-making relative to cancer health and clinical care. To measure the change in knowledge and learning, surveys, interviews, focus groups, data analytics and self-reports of education and attitudes over a period are documented and monitored.

As knowledge and learning is strengthened, it leads to changes in action or utilization, such as increasing primary and secondary cancer prevention and risk reduction behaviors, screening, and practices. There is also increased likelihood for research participation, and donation of biological samples for research, so that clinical needs can better serve minoritized and marginalized communities. Employing more community-focused approaches also increases the potential for community trust, partnerships, and community engaged research. Improved collaboration with invested partners will increase overall research scholarship in the field, with increased number of 1) underrepresented minorities (URMs) in biomedical research; 2) COE presentations, publications, and grants; 3) focused community reports; and 4) community-engaged and health equity research led by MCCCC investigators. These intermediate outcomes are tracked through documentation of participation, self-reported behaviors, and feedback from community partners. Teams led by the operations team are implementing evidence-based interventions with guided evaluation plans in context-specific CA settings to determine reach, impact, and value of programs and best practices for future directions.

MCCCC COE long-term outcomes

The final component of the logic model describes the long-term outcomes and impact of the MCCCC COE, which includes decrease in cancer morbidity, mortality, and disparities in the CAs measured over time by the cancer incidence, prevalence and mortality rates in CA communities. It is also expected that there will be an increase in the representation of minoritized and marginalized communities in clinical trials, biomedical research, and tissue donation to effectively address cancer health disparities in CAs. Additionally, increasing the representation of URMs in the biomedical research workforce expands diversity in research and clinical practice. Finally, the MCCCC COE is increasing global oncology research capacity in LMICs to address the burden of cancer globally.