Evidence-based health policy in Germany: lack of communication and coordination between academia and health authorities?

Epidemiology of systematic reviews

Health-care decision making should consider the best available evidence. In the spirit of evidence-based medicine, it is recommended to obtain such evidence by relying on high-quality systematic reviews (SRs).

The number of published SRs is growing exponentially: In 2010, Bastian et al. [1] published their remarkable study entitled “Seventy-five trials and eleven SRs a day: how will we ever keep up?”. Other authors have subsequently attempted to estimate the number of SRs [2,3,4], and while they sometimes differed in their prevalence estimates, they always agreed on the trend for exponential increase. The most recent such study found that nearly 80 SRs were published daily in 2019 [5].

Not surprisingly, this comes at the cost of overlapping SRs, i.e., multiple SRs dealing with the same or similar question [6, 7]. Replication is desirable to some extent [8], and multiple SRs on the same question reaching the same conclusion will increase confidence [9]. However, overlap leaves SR users with two major challenges. First, SRs can reach different conclusions [10,11,12]. Second, the reporting quality as well as the methodological quality of SRs still leaves a lot of room for improvement (e.g., [13]). Clearly spoken, there is a considerable amount of research waste in the conduct of SRs [14]. From a decision-maker’s perspective, this can be a very convoluted scenario. Should I use an already published SR? Is it reliable, methodologically sound, and does it address my specific research question? How do I choose one among multiple SRs? The risk of bias tool in systematic reviews can be helpful in answering some, but not all of these questions [15]. However, if one is not able or willing to rely on other authors’ SRs, there is no other choice than conducting a de novo SR, and by this potentially contributing to research waste. Notably, a recent review found that researchers make no or poor use of SRs when justifying and/or designing new studies, or when placing new study results in the context of existing similar research [16].

This article aims at encouraging a discussion on unintended duplication of systematic reviews or analyses by different health authorities and academia resulting from poor communication and coordination between the two worlds. For illustration, three examples are presented that raise the question of useful replication or research waste.

Health authority organisations in Germany

This section presents German health authority organisations, which is relevant for understanding the following three cases. The Federal Joint Committee is the highest decision-making body in the German health care system. It regularly commissions the Institute for Quality and Efficiency in Health Care (IQWiG) with systematically reviewing the evidence on drugs, diagnostic, or therapeutic procedures as a basis for their decision-making purposes. IQWiG is scientifically independent. There is a legal foundation in the German Social Code Book V (the law for the German public health insurance system) of this sharing of responsibilities. Decisions made by the Federal Joint Committee are sometimes subject of court proceedings. IQWiG’s rigorous approach has resulted in the recognition of the trustworthiness of their evidence reports by German jurisdictions. IQWiG preferably synthesises evidence from RCTs and other clinical trials according to their own methodology instead of relying on already existing evidence syntheses. This inevitably increases the number of SRs and thus risk of duplicate SRs. In addition, there is no pre-registration of IQWiG reviews in databases of SRs, such as PROSPERO (but review protocols are published on the website). As part of its reviews, IQWiG also searches for pre-existing SRs and refers to them in the results or even uses them instead of primary studies; however, IQWiG is required by law to perform independent assessments (and reviews) with participation of clinical or methodological experts and as a matter of fact, usually relies on primary trial results. The Institute for Quality Assurance and Transparency in Health Care (IQTIG) is responsible for statutory quality assurance in the health care system. The Federal Joint Committee commissions IQTIG for the development and implementation of quality assurance procedures. The Federal Office for Radiation Protection is responsible for the safety and protection against radiation damage to the environment and population health. Therefore, its mandate includes assessing health-related risks from radiation (e.g., X-ray diagnostics) to advise the Federal Ministry for the Environment and other health authorities.

Below, the authors present three cases of unintended, publicly funded, duplication.

Total knee arthroplasty and the minimum volume-threshold (first case)

One example for unintended duplication of evidence synthesis by academia and health authorities is related to “minimum volume standards” and the hospital volume-outcome relationship for total knee arthroplasties. There was no high-quality SR on this topic and thus no evidence for a minimum volume standard. Yet, German hospitals need to perform at least 50 total knee arthroplasties annually to get reimbursement for this procedure [17]. Therefore, the Institute for Research in Operative Medicine at Witten/Herdecke University received a grant notice in 2018 by the Ministry for Research and Education to conduct a SR. Although the review was registered on PROSPERO [18] and a protocol was published in 2020 [19], the Federal Joint Committee commissioned IQWiG to investigate the same question within a rapid report in 2021. Rapid reports by IQWiG aim to provide up-to-date information on current relevant topics and are comparable to rapid reviews. In the same year, the SR conducted at Witten/Herdecke University was published online [20], while IQWiG’s report was completed and published in 2022 without referencing the former [21]. With only minor differences in eligibility criteria, both reviews drew the same conclusions. On the one hand, the example can be seen as a case of unintended duplication of efforts because both reviews relied on the same set of primary studies, albeit the total number was lower in the IQWiG report due to stricter inclusion criteria. On the other hand, there have been many political debates on minimum volume standards and two reviews by independent institutions may strengthen the conclusions. In addition, different intentions and legal pathways may explain the case. Nevertheless, better communication and coordination between both institutions could increase synergies.

Lung cancer screening (second case)

Various randomized controlled trials (RCTs) on lung cancer screening in heavy smokers have been completed within the last years. In many of these studies, computed tomography (CT) was found to reduce lung cancer mortality. SRs and clinical guidelines therefore recommended low-dose-CT screening [22]. The topic was also picked up in Germany, but the regulatory framework is complex: In the first step, any new screening intervention that uses X-rays needs to be formally approved by the Federal Office for Radiation Protection. This requires a risk-benefit assessment including modeling of long-term radiation safety issues [23]. In the second step, any new screening intervention has to be assessed in order to decide on statutory reimbursement and establishment of a screening programme by IQWiG. Both evaluation steps include a systematic assessment of risks and benefits. Accordingly, two SRs on the same topic were prepared within a few years by official agencies—first the Federal Office for Radiation Protection, then IQWiG. In the case of lung cancer screening, both assessments were based on the same set of primary studies and resulted in nearly identical effectiveness results [24, 25].

Unintended duplication of efforts could potentially have been avoided if both agencies (Federal Office for Radiation Protection and IQWiG) had better communication processes in place, allowing them to join efforts. From a scientific perspective, it would require fewer resources either if both agencies collaborated directly, or if one agency built on the previous work by the other. However, legal reasons require that both assessments are done independently from each other. Issues of confidentiality and independence allow only minimal scientific exchange between both agencies. The current workflow is stipulated by national law, which follows the principle of independent assessments without specifically aiming at replication of results.

Transcatheter aortic valve implantation (third case)

Another example for unnecessary duplicate work in the context of evidence syntheses is the analysis of the relationship between volume and outcome in transcatheter aortic valve implantation (TAVI). In contrast to the other examples, it does not refer to multiple SRs but to primary data analyses.

Specifically, in 2021, IQWiG published an evidence report on the association between hospital volume and quality for TAVI [26]. The available evidence was not conclusive regarding all outcomes of interest. In addition, because no “German” study was included, it was doubted that the results were applicable to the German healthcare context. For this reason, the German Society of Cardiologists decided to verify the findings and gain additional insight for the German context. For this purpose, it was obvious that it would be best to use the German external quality assurance data, which cover all hospitals in Germany. The data can be analyzed by any researcher on reasonable request and are kept by IQTIG. A committee of the Federal Joint Committee must approve all proposals. The protocol by the German Society of Cardiologists for the analysis was submitted to IQTIG in May 2022 and accepted in July [27]. In the same month, the Federal Joint Committee itself commissioned the IQTIG to perform an analysis on the same question [28]. The analysis appears to be an unnecessary duplication of research because the same data are used and consequently, differences can arise only because of differences in the analysis methods. Both analyses are not yet published. The Federal Joint Committee can commission its own analyses instead of relying on already existing evidence syntheses to obtain the analyses that perfectly fits its decision problems. Therefore, this duplication can be considered to be unintended in the sense that it was not a planned replication.

In the sense of evidence-based research, the example may be viewed positively, as it was performed in response to a research gap identified in an evidence synthesis. However, it illustrates that research to fill such knowledge gaps should be coordinated.

留言 (0)

沒有登入
gif