1 BACKGROUND

Despite the rapid growth of computable biomedical knowledge in the United States,1 there is a widening gap of health disparities2 and health outcomes are worse than other countries that invest less in health innovation.3 The impacts of the COVID-19 pandemic have brought into focus the differences in health and health care between population groups. While the number of potential treatments and evidence base is increasing, to be widely disseminated and implemented in practice, there are several challenges to overcome including volume, relevance to patients, and need to adapt to workflows and electronic health record (EHR) technologies used where health-related decisions are made.4, 5

To be rapidly and widely disseminated for application, actionable knowledge needs to be distributed in easily usable and readily implementable formats, that is, computable biomedical knowledge (CBK). CBK, such as predictive models, rules, alerts, clinical pathways, or data visualizations, is necessary for the interventional approach of a learning health system. The mobilization of CBK can result in rapid mass access to computable knowledge, which has the capacity to improve the health of individuals and populations on a large scale.4 We believe that sharing CBK artifacts widely can only be achieved if they are findable, accessible, interoperable, reusable, and trustable (FAIR+T).6

While work has proceeded in this space for many years, the movement to mobilize computable biomedical knowledge (MCBK) was conceived 4 years ago by several thought leaders as well as a diverse multistakeholder community of interest. The MCBK community aims to achieve improved health in diverse settings by widely sharing knowledge in a computable format.7, 8 In this report, we present a summary of the Fourth Annual MCBK public meeting held virtually on July 20 to July 21, 2021.

2 MEETING AND PARTICIPANT INFORMATION Due to the continuing COVID-19 global pandemic, the meeting was held for the second year as an online interactive conference. About 150 people were registered before the event and 118 unique participants joined virtually, representing the following types of organizations: Universities/Academic Medical Centers (n = 63 [53%]) Commercial/Industry and Consultants (n = 10 [<1%]) Multidisciplinary Clinicians (n = 25 [21%]) Government (n = 7 [<1%]) Other (n = 13 [11%]) Students/Fellows (n = 5 [4%]) 3 MEETING STRUCTURE AND OVERVIEW

A multidisciplinary and multistakeholder steering committee guided the selection of topics and activities. The virtual meeting included remarks from national leaders, panel presentations, and a lightning round of poster sessions and technical demonstrations. There were also breakout sessions for work groups and exposition rooms for poster presentations. The meeting agenda, list of speakers, registered participants, and presentations are available at www.mobilizecbk.org. Slides and videos for all talks are available at https://mobilizecbk.med.umich.edu/news-events/annual-meetings/2021-meeting.

An in-person meeting was not possible due to the COVID-19 pandemic and restrictions on travel and interpersonal contact. A professional AV service was used to set up a virtual platform, Hopin, for streaming the meeting live to viewers. Central meeting moderators led the two-day conference from a studio in Ann Arbor, Michigan. The Hopin technology enabled individual speakers and groups of presenters to share with the audience. Attendees could submit questions in the chat and question-and-answer windows on Hopin, where in-studio central meeting moderators would relay them to speakers for answering. The Hopin platform, which enabled audience participation and interactivity, was also used for breakout meetings with the different work groups and for the poster session and social hour aspects of the meeting.

4 MEETING SPEAKERS AND CONTENT 4.1 Welcoming address and remarks from national leaders

Dr. Rachel Richesson and Dr. Charles (Chuck) Friedman, MCBK Steering Committee co-chairs, opened the meeting with brief remarks on the fundamental principles of MCBK and presented an overview of the 2-day agenda. Meeting attendees received a video9 in advance of the meeting that related the origin of the MCBK movement. Drs Richesson and Friedman presented the meeting goals: to chart the future of MCBK as a membership organization, to strengthen the foundation of shared recognition and principles for mobilizing CBK, to advance work group action plans, to generate new ideas and identify opportunities for future collaboration, and to share reports of ongoing work.

Dr. Deborah McGuinness, Tetherless World Senior Constellation Chair and Professor of Computer, Cognitive and Web Sciences, Rensselaer Polytechnic Institute, gave an opening keynote address emphasizing the importance of moving toward a world with “more” computable knowledge.10 As a leader in computable knowledge and representation, Dr. McGuinness outlined three areas that could increase the impact of insights the community could get from computable knowledge. She focused first on recommender systems and cognitive assistants that are already in use today and emphasized that these systems could have improved impact with enhanced explanation and usability, particularly in biomedical applications. Dr. McGuinness shared how emerging personal knowledge stores could make more of a difference by way of enhanced access, control, interoperability, and tooling infrastructure. Finally, she addressed how meta descriptions aimed at use and reuse of knowledge resources are being built for longevity and could improve impact with increased standards for meta descriptions and methodologies. Dr. McGuinness related that the meta descriptions, which encode things like embedded assumptions and use cases, are in the early emerging stages but will be important in enabling customization of harmonized data sets.

Dr Matthias Kretzler, Warner-Lambert/Parke-Davis Professor of Internal Medicine/Nephrology and Computational Medicine and Bioinformatics at the University of Michigan shared his perspectives on research in scalable computable knowledge for integrated systems biology.11 Dr. Kretzler emphasized the need to “get personal and precise” from data to knowledge so that meaningful information can be extracted from study participants in research. He shared the benefits of having accurate diagnosis and prognosis to be able to provide targeted treatment and improved outcome, by presenting examples from two globally distributed research networks for rare diseases, nephrotic syndrome study network (NEPTUNE), and the Kidney Precision Medicine Project (KPMP).

Dr. George Strawn, Director Emeritus on the Board of Research Data and Information for the National Academies of Science, Engineering and Medicine presented the Day 2 keynote titled, perspectives on the emerging field of computable knowledge. Dr. Strawn, a former Knowledge Officer, spoke about the importance of FAIR digital objects and emphasized what he described as Computable Knowledge of Type I (computer-enabled human know what) and Computable Knowledge of Type II (computer know how). He said if we move toward the ability to automate knowledge acquisition, it would be a significant step in creating an “all science knowledge base enabling better interdisciplinary open science.12”

Dr. Mark Musen, Director of the Stanford Center for Biomedical Informatics Research addressed the subject of Metadata and the CBK life cycle. Describing metadata as “a love note to the future…and present,” Dr. Musen emphasized that the development of metadata for computable knowledge should be based on the FAIR guiding principles. He said the categories outlined by the MCBK Standards5 group gives us a framework for the structure of the metadata we need to get started. Dr. Musen emphasized that the hard work will be taking the categories assigned by the MCBK Standards group and coming up with descriptors for the metadata that we need to create online biomedical knowledge to make it FAIR.

4.2 Panel presentations

A MCBK Workgroup panel moderated by Dr. Richesson addressed the progress each group has made in mobilizing CBK. The Sustainability and Inclusion Work Group co-chaired by Gerald (Jerry) Perry and Terrie Wheeler spoke about the group's focus on advocacy through affiliations, scholarship, education, and research. The working group developed an association with the American Association of Health Sciences Libraries (AAHSL) and created a memorandum of understanding with the idea of replicating this for other groups that would like to form an association with MCBK. The working group hopes to develop a web-based guide of resources on health equity and CBK, along with a sustainability and inclusion “roadmap for research.” If mobilized according to principles such as those in the MCBK Manifesto, CBK can advance health equity and reduce disparities. However, if CBK ends up only in the hands of the few, it risks exacerbating disparities and further entrenching the status quo power structures.

The Standards and Technical Infrastructure (now Standards and Infrastructure) Work Group co-chaired by Drs Bruce Bray, Robert (Bob) Greenes, and Jamie McCusker highlighted the group's recent publication on categorizing metadata.5 The working group is also editing a white paper on guiding principles for technical infrastructure to support computable biomedical knowledge. The co-chairs outlined the group's intention to develop a collaborative mechanism across MCBK working groups and the connections it hopes to develop with other semantics and bio collaborations.

The Policy and Coordination to Ensure Quality and Trust Work Group discussions co-chaired by Drs Jodyn Platt and Blackford Middleton highlighted their focus over the last year on identifying principles and best practices to promote trust in digital knowledge repositories. The co-chairs outlined goals focused on four areas: (1) identifying common principles and practice guiding governance of repositories, (2) identifying common metadata practices that ensure trustworthiness of artifacts, (3) developing measures of trustworthiness, and (4) using findings to inform CBK policy.

4.2.1 CBK: Responding to COVID-19

A two-person panel13 moderated by Dr. Richesson discussed using CBK to respond to COVID-19. Mr. Brian Alper, founder of Dynamed presented the Fast Evidence Interoperability Resources (FEvIR), which mobilizes scientific knowledge online. Mr. Alper demonstrated live the ability of the FEvIR knowledge platform to transform all the data that appeared in a PubMed article into FHIR resources.

Dr. Jerry Osheroff described how CBK-powered learning health systems can be leveraged to achieve the quintuple aim of healthcare-improved clinical experience, better outcomes, lower costs, improved patient experience and improving sick care business processes. Dr Osheroff described the lack of computable, interoperable information and tools and emphasized how certain developed tools, like FEvIR, can be crucial in highlighting the critical need to apply the latest clinical evidence and guidance and cases such as the COVID-19 pandemic.

4.2.2 Mobilizing CBK worldwide

As part of the Annual Meeting goal to grow collaboration and build relationships with entities of similar interests, three international speakers described their organization's synergies with the MCBK movement.14

Dr. Phillip Scott of University of Portsmouth described how the United Kingdom (UK) team presents the MCBK message as a set of algorithms that can service multiple use cases. He said the main conclusion from the first MCBK UK meeting was the significance of CBK to future health systems, which should be supported and developed with cross-sector support from informatics and clinical experts.

Dr. Guilan Kong of Peking University described the mobilization of CBK in China and addressed the opportunities and challenges computable knowledge presents in her country. She said although CBK is a new concept in China, there is a recognized need and there is acknowledgment that it would present an opportunity to address the knowledge representation and dissemination needs from the large number of Chinese publications and accumulated data in medicine.

Dr. Enrico Coiera of Macquarie University in Australia spoke of the need for computable evidence synthesis. He referred to a recent publication in JAMIA by the Australian team, on replication studies in clinical decision support, which addressed frequency, fidelity, and impact.

4.2.3 The future of MCBK

Dr. Friedman and Dr. Richesson introduced the intention of forming MCBK into a formal membership organization with an interim home at the University of Michigan. They both addressed how far the movement has come since its first meeting in 2017 and articulated a future organization mission and vision that is guided by the MCBK Manifesto.

Dr. Douglas Van Houweling, Professor Emeritus at the University of Michigan presented a draft document that described a rationale for MCBK evolving into a membership organization. Dr. Van Houweling highlighted the MCBK Manifesto vision and described the proposed goals of a membership organization, convening and organizing a community, setting directions for CBK, and acting to mobilize CBK.

4.3 Poster session

Twenty-one posters were presented via an online lightning round on Day 1 of the meeting. There were three3 technical posters and eighteen15 project posters. Poster authors also expanded on their posters at a designated poster session at the end of Day 1. Each poster was assigned a virtual exposition room and presenters were able to expand on the posters during the session.

Of the 21 posters, 4 (19%) came from commercial entities, 14 (67%) from academia, 1 (5%) from government, and 2 (9%) from standards development organizations. Collectively, the posters represented the perspectives of knowledge developers, disseminators, and users. Poster abstracts from the meeting are included in this issue and digital posters can be viewed here: https://mobilizecbk.med.umich.edu/news-events/annual-meetings/2021-meeting

4.4 Tribute

Dr. Friedman, Dr. Richesson, and Dr. Valerie Florance, Scientific Director for Intramural Research at the National Library of Medicine (NLM) of the United States National Institutes of Health (NIH), paid tribute16 to the late Dr. Milton Corn, Deputy Director for Research and Education at NLM. Dr. Corn was a longtime supporter and friend to the MCBK community. Several meeting attendees offered words of remembrance for the significant role Dr. Corn played in the advancement of CBK. There were many personal testimonies to the ways Dr. Corn had consistently put other people, as well as causes likely to protect the health of other people, before himself.

5 WORKGROUP ACTION SESSIONS AND ACTIVITY

The speakers described above provided background, vision, and motivation for meeting participants, who were charged to advance the MCBK vision through the three (formerly four) MCBK work groups. Due to similarities and alignment of research efforts, the Standards and Technical Infrastructure workgroups merged to form the Standards and Infrastructure workgroup in early 2021. One breakout session (2.5 h on Day 2) was designated as a Work Group Action Session. The work groups and their co-chairs, scope, and discussions are summarized below.

The Standards and Infrastructure Work Group (SWG), led by Drs. Robert Greenes, Bruce Bray, and Ms. Jamie McCusker, attracted a number of diverse members from both academia and industry who are engaged in projects that address the technical requirements and relevant standards for sharing CBK at scale and applying in specific—often clinical—settings. The group reviewed accomplishments (including an articulation of infrastructure principles and paper6 describing categories of metadata for CBK) for new members and also discussed areas for future work, including detailed metadata requirements for collaboration across the entire CBK lifecycle and the exploration of existing metadata standards (eg, W3C DCAT) from web and biological research communities. The group also discussed strategies to create synergy with high-profile knowledge deployment initiatives such as the AHRQ evidence-based Care Transformation Support (ACTS), COVID-19 Knowledge Accelerator (COKA), Evidence-Based Medicine Knowledge Assets (EBMonFHIR), and Logica. The breakout discussions also recognized the growing need for tooling to collaborate and coordinate across the 3 MCBK workgroups to ensure that the MCBK community can work efficiently to mobilize CBK to advance our response to COVID-19 and health problems as they emerge.

The Policy and Coordination to Ensure Quality and Trust Work Group discussions were led by co-chairs Jodyn Platt and Blackford Middleton. The Trust and Policy Working Group (TPWG) used the breakout time to review results of a survey they had conducted of CBK repositories, and their current practices related to the TRUST Principles for data repositories: (1) Transparency, (2) Responsibility, (3) User Focus, (4) Sustainability, and (5) Technology.17 Additional questions in the survey provided general descriptive information about the organizations themselves, such as when they were founded and their funding models, and about CBK artifacts, such as how their CBK artifacts abstracted biomedical knowledge for their intended users. The TPWG discussion reflected on the preliminary survey results to consider (1) the implications of findings for the CBK ecosystem and its stakeholders, (2) salient opportunities and challenges for the field, and (3) key research questions moving forward.

The Sustainability for Mobilization and Inclusion Work Group, chaired by Terrie Wheeler and Jerry Perry, began with a quick review of plans for the session followed by distinct, focused conversations in two key areas of Group activity: Advocacy Through Scholarship and Advocacy Through Education. For the former, a small subset of the Group who have been working on a “thought piece” for publication on health equity and computational biomedical knowledge (CBK) gave a quick update and then broke off for a side conversation advancing their ongoing work. The rest of the attendees discussed two ideas intended to advance the Group's engagement strategy through education: creating educational video vignettes on CBK topics and organizing a mid-year panel of experts on CBK themes to enhance the momentum of the Group's work. The vision for the video vignettes project is to develop short recordings that could be used to explain or promote CBK-related themes and topics, resulting in mobile, flexible, reusable learning content that could be deployed in a variety of contexts addressing, for instance, some of the challenges in explaining what CBK is and why it matters. To advance this topic, staff from the University of Michigan A. Alfred Taubman Health Sciences Library's video production team shared recommendations for video production values. The Group brainstormed topics to include, along with suggestions for speakers and content experts. Among the topics considered: ontologies, what they are, and how health sciences libraries contribute to their production, use and accessibility; the CARE Principles for Indigenous Data Governance18 and how they contrast with FAIR Principles15; and metadata for CBK vs metadata for data and other knowledge objects. The vision for the expert panel project is to generate panel-based programming that could be used to advocate, market, and educate about CBK while also helping us to maintain the momentum of Group work outside of the annual meetings. This programming could also be a way to advance tangible engagement with other MCBK Work Groups or other entities such as the Association of Academic Health Sciences Libraries (AAHSL), American Medical Informatics Association (AMIA), among others. As with the video vignettes, attendees brainstormed on the topic of potential speakers to be featured as experts in the field. Among the topics considered: a panel featuring real-life uses and applications of CBK and a session on health equity, bias, machine learning, and artificial intelligence.

6 CLOSING AND REFLECTIONS

The meeting closed with reflections from Dr Peter Embi, President and CEO of Regenstrief Institute and Dr. Leslie McIntosh, Executive Director for Research Data Alliance. Dr. McIntosh, a former co-chair of the MCBK Technical Infrastructure workgroup, recapped the common themes of the speakers across the two days. She offered her thoughts on the use cases to present the need, capabilities, and importance of informed curated knowledge, which should be balanced by equity, bias, and fairness. Dr. McIntosh also emphasized the need for knowledge sharing across geographical borders and encouraged the three MCBK workgroups to build upon the organizational structure and goals outlined for a future MCBK membership organization and said there is a need for “trusted” knowledge in this age of misinformation.

Dr. Embi, a member of the MCBK Steering Committee, highlighted some themes that he thought emerged from the meeting. He described the momentum and acceptance of MCBK as encouraging, especially the understanding of MCBK in context. He emphasized the linking of CBK tools and activities to the patient journey and is impressed by the growing stakeholder movement. Dr. Embi sees MCBK as a key “enabler” of the Learning Health System and says there is early evidence of impact, referencing how MCBK was represented in tackling the COVID-19 pandemic. He said there is still a need to develop a “holistic playbook” to operationalize MCBK and says there is opportunity to develop and demonstrate best practices in a way that impacts health and health care in an ethical and equitable way.

7 NEXT STEPS

MCBK continues to fill an important but broad niche based on the diversity of meeting attendees. Work group chairs and members plan to continue their activities into the next year and support plans for subsequent public meetings.

The University of Michigan will continue to provide communications and logistical support for MCBK workgroups and their members. Plans for a Fifth Annual MCBK public meeting for Summer 2022 are underway. MCBK is an open and inclusive community. Anyone that is interested in joining an MCBK work group may sign up here: http://mobilizecbk.org/

ACKNOWLEDGMENTS

This meeting or report could not have been possible without the significant efforts of MCBK work group co-chairs (listed alphabetically): Bruce Bray, Robert (Bob) Greenes, Jamie McCusker, Blackford Middleton, Gerald (Jerry) Perry, Jodyn Platt, and Terrie Wheeler.

The authors wish to thank Janelle Burleigh and Yunzhu (Helen) Pan for their significant contribution to the planning and management of the meeting, and for their ongoing and meticulous project management support for the MCBK initiative. We also wish to recognize important contributions from others at the University of Michigan, including: Peter Boisvert, Lisa Ferguson, Allen Flynn, Charles Friedman, Brooke Raths, Joshua Rubin, and Doug Van Houweling.

The authors gratefully acknowledge the support of the University of Michigan as well as the contributions of all conference speakers, moderators, rapporteurs, and participants. The authors thank the NIH and NLM for supporting the meeting. The opinions expressed here represent those of the authors and do not represent official views of the University of Michigan or of NIH or any of its institutes and centers.

Participants paid modest registration fees to cover costs for the online meeting broadcasted by Russell's Video Services in Ann, Arbor, Michigan. The meeting costs were supported in part by the Department of Learning Health Sciences at the University of Michigan Medical School.

CONFLICT OF INTEREST

The authors have no competing interests to declare. At the time of this meeting, Blackford Middleton was employed by Apervita, Inc., which provides a platform and marketplace to democratize healthcare data and analytics.

REFERENCES

1Larsen PO, von Ins M. The rate of growth in scientific publication and the decline in coverage provided by science citation index. Scientometrics. 2010; 84(3): 575- 603. 2Dwyer-Lindgren L, Bertozzi-Villa A, Stubbs RW, et al. Inequalities in life expectancy among US counties, 1980 to 2014: temporal trends and key drivers. JAMA Intern Med. 2017; 177(7): 1003- 1011. 3 APHA. Health Rankings. From the Nation's Health: What Makes Communities Healthy? New Rankings Showcase Successes. Washington, DC: American Public Health Association; 2019. https://www.apha.org/topics-and-issues/health-rankings 4Friedman CP, Flynn AJ. Computable knowledge: an imperative for learning health systems. Learn Health Syst. 2019; 3(4):e10203. 5 MCBK. Mobilizing Computable Biomedical Knowledge (CBK): A Manifesto: Mobilizing Computable Biomedical Knowledge; 2018 (updated October 7, 2019). https://medicine.umich.edu/sites/default/files/content/downloads/MCBK%20Manifesto%20Ver%2010.7.18.pdf. 6Alper BS, Flynn A, Bray BE, et al. Categorizing metadata to help mobilize computable biomedical knowledge. Learn Health Syst. 2021;e10271. doi:10.1002/lrh2.10271 7Flynn A, Iott B, Richesson R. Metadata Standards for Computable Bio- Medical Knowledge (CBK). Ann Arbor, Michigan: Ann Arbor University of Michigan; 2018. 8 Mobilizing Computable Biomedical Knowledge. Conference Overview. https://deepblue.lib.umich.edu/bitstream/handle/2027.42/140789/Overview%20of%20MCBK%20Conference%20Event.pdf?sequence=1&isAllowed=y 9Friedman C, Richesson R. MCBK 2020 Welcome and Opening Remarks. https://www.youtube.com/watch?v=x6YN0QmU9SI 10McGuinness D. MCBK 2021. Day 1 Opening Keynote. https://www.youtube.com/watch?v=kmpQsUoo4YM 11Kretzler M. MCBK 2021. Day1 Remarks. https://www.youtube.com/watch?v=FtjyeF8Qznc 12Strawn G. MCBK 2021. Day 2 Keynote. https://www.youtube.com/watch?v=71rz1Iz5i4k 13Alper B and Osheroff B. MCBK 2021. Day 1 panel. https://www.youtube.com/watch?v=6G18pxdV42c 14Cong G, Coiera E, Scott P. MCBK 2021. Day 1 Panel discussion. https://www.youtube.com/watch?v=2YLvAVeGbEU 15 FAIR Principles: F – Findable; A – Accessible; I – Interoperable; and, R – Reusable. See: https://www.go-fair.org/fair-principles/. Last viewed Sept. 24, 2021. 16Florance V, Friedman C & Richesson R. MCBK 2021. Day 1 Tribute. https://www.youtube.com/watch?v=JWDMo4Cm8KI 17Lin D, Crabtree J, Dillo I, et al. The TRUST principles for digital repositories. Scientific Data. 2020; 7(1): 1- 5. 18 CARE Principles for Indigenous Data Governance: C – Collective benefit; A Authority to control; R – Responsibility; and, E – Ethics. See: https://www.gida-global.org/care. Last viewed Sept. 24, 2021.