Introduction

Transcatheter aortic valve implantation (TAVI) has established itself as a valuable alternative to surgical aortic valve replacement.1 2 Despite TAVI’s success, stroke is still considered a critical procedure-related complication.3 Considerable efforts have been made to reduce procedure-related stroke rates in general, and to diminish early stroke in particular. Cerebral embolic protection (CEP) devices can protect the brain from embolic debris during the procedure and have already been applied for the past decade, with a varying adoption rate.4

Several randomised trials have addressed using CEP during TAVI to prevent stroke. However, many clinical trials were hampered by small sample sizes and low outcome rates, and were underpowered to detect statistically significant differences. Of note, these limitations may also extend to pooled analyses of literature, and previously applied statistical analyses could also not evaluate a potentially clinically relevant CEP treatment effect.5 6

Bayesian inference provides the opportunity to incorporate prior evidence from literature and estimate the posterior probability of a treatment effect across a wide spectrum of effect size thresholds, including the minimal clinically relevant difference (MCID).

Therefore, we aim to perform a systematic review and meta-analysis of randomised CEP trials under the Bayesian framework, focusing on a clinically relevant reduction in the rate of disabling stroke.

MethodsProtocol registration

This study is preregistered in the PROSPERO Database (CRD 42023407006, dated 21 March 2023) and adheres to the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. Furthermore, we report all items for a Bayesian analysis as identified by the ROBUST criteria.7

Search strategy and study inclusion

We performed a systematic search that was applied to three electronic databases (ie, PubMed Central, MEDLINE (through PubMed), Embase and the Cochrane Library) from 1 January 2000 until 1 March 2023 (online supplemental material 1).

Studies are included when randomising TAVI patients to a CEP device versus standard treatment and at least reporting on stroke. If cohorts overlap, the corresponding author of that study is contacted to provide the outcome details of randomised patients. Studies are excluded when only reporting on patients undergoing a non-transfemoral approach. The search and study selection are independently performed by two authors (SH, PAV). Non-randomised studies excluded in the last phase will be evaluated for their eligibility to serve as an informed literature-based prior.

Data extraction

A predefined worksheet is presented in online supplemental material 2. Data extraction is performed independently by two authors (SH, PAV), and any disagreement will be resolved by a consensus.

Risk of bias in individual studies

The Revised Cochrane Risk of Bias Tool for randomised trials will be applied to evaluate the risk of bias. This assessment is performed independently by two authors (SH, PAV).

Outcomes

The primary outcome of the current meta-analysis is disabling stroke, given its significant impact on prognosis and health-related quality of life.8 Disabling stroke was uniformly defined as a score of 2 or higher on the Modified Rankin Scale at 90 days.9 10 The secondary outcome is all stroke,9 10 given its important impact on other clinical outcomes.11 The evaluated safety outcomes include life-threatening bleeding and major vascular complications.9 10

All pooled outcome measures are expressed as relative risks (RRs) with corresponding 95% credible intervals (CrIs) or the natural logarithmic functions of the RR with 95% CrIs. These RRs are converted to absolute risk differences (ARDs) and numbers needed to treat (NNTs), as proposed by the Cochrane Collaboration.12 These (log) RRs, ARDs, NNTs and accompanying distributions will be calculated using the formulas presented in online supplemental material 3.

Using Bayesian inference, we will study the posterior probabilities of any benefit (ie, RR <1.0) and the probabilities of RR <0.75, RR <0.50 and RR <0.25. Furthermore, the posterior probability of the MCID will be estimated.

Minimal clinically important difference

Several MCIDs for ischaemic stroke prevention and treatment have been proposed based on comprehensive surveys, ranging between 1% and 10% ARD.13 14 We objectively derived the MCID (1.1% ARD; 11–15 patients saved in a population of 1000) for disabling stroke of a recent expert consensus statement,13 translating to an NNT of 91 patients to prevent one disabling stroke.

Statistical analysis

The principal meta-analysis is performed under a Bayesian framework on the log RR scale using a vague prior. This vague prior does not assume a difference between both groups and is weakly informative (mean (μ) 0 and SD (σ) 2). As randomised trials may not always accurately reflect the specific patient population undergoing the study intervention,15 we will incorporate an informed literature-based prior (the Society of Thoracic Surgeons-Transcatheter Valve Therapy (STS-TVT) registry)4 to verify the robustness of the results (μ=−0.28, σ=0.09). For sensitivity purposes, a second informed literature-based prior is also applied, which is derived from the National Inpatient Sample (NIS) Database (μ=−0.40, σ=0.14).16 The rationale and details for selecting these priors can be found in online supplemental material 4. To account for the observational character of these priors, both were also downweighted to 75% and 50%. An averaged model is used to report the eventual effect size. Between-study heterogeneity is tested and expressed by the I2 metric and τ.

Posterior probabilities are estimated using Markov chain Monte Carlo sampling (3 chains, 10 000 saved iterations per chain), with JASP (JASP team, 2023, V.0.17.1, for Mac, Amsterdam University, Amsterdam, the Netherlands). All other analyses are performed in R studios (R Foundation for Statistical Analysis, Vienna, Austria), using the ‘meta’ and ‘dmetar’ software packages and Meta-Essentials.

Publication bias

Publication bias will be assessed for the primary outcome (disabling stroke) and evaluated both visually (funnel plots) and statistically (Egger’s test, for which p<0.05 indicated statistical significance).

Patient and public involvement

This study was performed without involvement of patients.

ResultsStudy inclusion

The systematic search yielded 996 hits, of which n=429 were excluded because of duplication or other reasons (n=10). Consequently, 557 studies were screened based on title and abstract, after which 31 underwent full-text assessment (figure 1 presents reasons for exclusion).

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 flow chart for study inclusion, including prior selection.

A large contemporary registry was selected as a primary informed prior,4 while we elicited a secondary informed literature-based prior for sensitivity purposes.16

Study characteristics

All seven included studies comprised randomised controlled trials and were published between 2015 and 2022.17–23 Patients randomised to the intervention arm underwent CEP with either a shield-based17 21 22 or filter-based device.18–20 23 The REFLECT II trial incorporated roll-in patients for the safety analysis and randomised control patients from REFLECT I for the primary efficacy analysis. Therefore, the corresponding author of these trials was contacted (AJL), who provided the outcomes of actually randomised patients. Table 1 presents these study characteristics.

Table 1

Overview of the seven included trials including study and patient characteristics, perioperative antithrombotic management and procedural factors

Patient characteristics

In total, 3996 patients were included in this pooled analysis (CEP n=2126, control n=1870 patients). Patient and procedural characteristics, and antithrombotic regimens, are presented in table 1.

Risk of bias

The risk of bias assessment is presented in online supplemental material 5. The risk of bias ranged from ‘some concerns’ to ‘low risk’. These concerns mainly arose from bias in the selection of the reported result. This high-risk bias was corrected by contacting the corresponding author of the REFLECT II trial.

Disabling stroke

The primary outcome was reported by all studies (n=7 studies, 3996 patients). Under a vague prior, the estimated posterior mean log RR is −0.59 (95% CrI −1.29; 0.17, figure 2A, I2=0%, τ=0, suggesting between-study heterogeneity is unlikely). This translates to a median RR of 0.56 (95% CrI 0.28; 1.19), with the estimated posterior probability of any benefit (RR <1.0) being 94.4%. Other posterior probability estimations and full posterior probability distributions are presented in figures 3 and 4 and online supplemental material 6.

Figure 2

Bayesian meta-analysis of disabling stroke under a vague prior (A) and an informed literature-based prior (B). Effect size is expressed as the log of the RR and 95% CrI, assuming a normal probability distribution of the Bayesian data. Heterogeneity I2=0%, τ=0. CEP, cerebral embolic protection; CrI, credible interval; RR, relative risk.

Figure 3

Probability density function for disabling stroke (A) and all stroke (B) under a vague and an informed literature-informed prior. The percentages represent the AUC (and therefore the probability) to the left of that specific threshold. AUC, area under the curve; CEP, cerebral embolic protection; RR, relative risk; TVT, transcatheter valve therapy.

Figure 4

Full posterior probability distribution for disabling (A) and all stroke (B), under a vague prior. CEP, cerebral embolic protection; RR, relative risk.

The pooled absolute risk of disabling stroke in the control group is 1.28% (95% CI 0.86% to 1.90%, also termed the assumed control risk (ACR)12). The estimated 44% RR reduction (RR 0.56) results in a 0.56% absolute risk reduction (100×0.0128×[1–0.56]) and an NNT of 179 treated patients to prevent a disabling stroke (table 2). Sensitivity analyses for pooled ARDs are presented in online supplemental materials 7 and 8.

Table 2

Absolute risk differences (ARDs) and numbers needed to treat (NNTs) to prevent a (disabling) stroke with CEP+, facilitating interpretation of clinical relevance

Under the primary informed literature-based prior, derived from the STS-TVT registry,4 the posterior mean log RR is −0.30 (95% CrI −0.46; −0.14), translating to a median RR of 0.74 (95% CrI 0.63; 0.87), with a 99.9% posterior probability of any effect of CEP on disabling stroke (figures 3 and 4). Based on this relative reduction and the 1.28% ACR, an absolute risk reduction of 0.33% is estimated, resulting in an NNT of 303 (table 2). Online supplemental material 9 presents the sensitivity analysis for the alternative informed literature-based prior (NIS Database), showing similar results (mean log RR −0.43, 95% CrI −0.69; −0.17, translated median RR 0.65, 95% CrI 0.50; 0.84, NNT 222). Of note, these results did not differ when both informed priors were downweighted to 75% and 50% (online supplemental materials 9 and 10).

Minimal clinically important difference

Based on expert consensus,13 an MCID of 1.1% ARD was deemed relevant (NNT 91). We estimated the posterior probability of the MCID using the vice-versa conversion of RR to ARD (with the ACR), as presented in table 2. The posterior probability of the MCID is 0.01% under a vague prior and 0% under the informed literature-based prior, which indicates how a clinically meaningful effect of CEP on disabling stroke is considered improbable. These findings were confirmed under the informed NIS Database sensitivity prior (online supplemental material 9).

All stroke

As a secondary outcome, all stroke (including non-disabling stroke) was evaluated under a vague and the informed literature-based prior (online supplemental materials 11 and 12, all studies). The posterior mean log RR is −0.10 (95% CrI −0.43; 0.25) and −0.24 (−0.39; −0.09) under both priors. The posterior probability of a clinically meaningful difference is 6.4% and 7.0%, respectively (table 2). Of note, the pooled absolute risk of all stroke in the control group was 3.64% (95% CI 2.88% to 4.58%).

Safety outcomes

Not all studies reported life-threatening bleeding and major vascular complications. Nevertheless, these complications were relatively infrequent (online supplemental material 13).

Discussion

The current meta-analysis comprises the largest pooled analysis of randomised controlled trials on the topic of CEP in transcatheter aortic valve replacement (TAVR). Furthermore, this analysis is the first to apply Bayesian inference to the difference between statistical significance and clinical relevance in this matter. The key results of our analysis are threefold: (1) there is a high probability of a beneficial effect of current CEP strategies in reducing disabling stroke after TAVI; (2) it is improbable that current CEP strategies lead to a clinically relevant reduction of stroke; (3) this inability seems to be mainly driven by the low rate of disabling stroke during TAVI.

Implications

The latter finding is probably the most reassuring for patients and clinicians. Since the conception of TAVI, stroke has arguably been its most feared complication. The results of a contemporary real-world registry suggested a stable annual stroke rate between 2011 and 2017.3 However, when comparing the stroke rates in the early years of TAVI in randomised high-risk patients (3.3–4.9%)24 25 to the lower-risk trials (0.6–3.4%),26 TAVI seems to have become a progressively safe procedure. Of note, the trials differed in stroke definitions and ascertainment. Our findings also confirm this contemporary procedural safety regarding stroke risk. We found a pooled 1.28% absolute risk of disabling stroke and 3.64% of all stroke in the control group.

When the event rate is relatively low, a study requires a considerable sample size to detect a statistically significant difference between two treatment groups. However, Bayesian inference allows the incorporation of external evidence through prior elicitation (potentially reducing the required sample size). It also facilitates the estimation of the posterior probability of various treatment effect thresholds, including a clinically relevant treatment effect.

This contrast between statistical significance and clinical relevance is underlined in the current pooled analysis under the Bayesian framework. Indeed, our results indicate a high probability of any beneficial CEP treatment effect on the outcome of disabling stroke (exceeding 94% under various priors, including the vague prior). Nevertheless, an ARD of 0.56% was most likely, which means 179 patients need to be treated with CEP to prevent one disabling stroke. It remains a subjective matter whether such an NNT is clinically relevant—or not. Therefore, we incorporated the MCID from published literature derived from expert consensus (ARD 1.1%, NNT 91).13 When estimating the posterior probability of this consensus-based MCID, a clinically relevant difference was deemed improbable (0–0.1% probability).

Particularly in contemporary cardiovascular trials, the generalisability of the studied population is increasingly questioned, as >70% of randomised studies exhibit relevant disparities between the sample population and typical real-world patients undergoing the procedure.15 27 The differences between the results under a vague prior (NNT 179) and the analysis under the informed literature-based priors (NNT 303 and 222) may be explained by this phenomenon. As such, the results under the informed literature-based prior (influenced by real-world registries) may be most representative of the expected outcomes of CEP in the general population.

Based on these findings, the widespread application of current CEP in all patients undergoing TAVI is not straightforward, as reflected by the low adoption in clinical practice. We did not observe relevant safety issues (as demonstrated by the low rates of life-threatening bleeding and major vascular complications). Still, the use of current CEP comes at a financial cost. Several studies regarding the issue of cost-effectiveness are ongoing. Therefore, a clinically meaningful effect should be interpreted in the context of the complication and this is ultimately subjective. In addition, the MCID may change over time, with the advent of new CEP devices and a reduction in complications and costs. Finally, this should also be weighed against the high societal and medical costs of disabling stroke, particularly in low-risk patients with a prolonged life expectancy.

Future directions

The British Heart Foundation PROTECT-TAVI trial is underway,28 which aims to include >7000 patients and expects a 33% RR reduction in all stroke, based on a foreseen 3% prevalence of stroke in the control group (1% absolute risk reduction, NNT 100). Although we encourage such rigorous trials to be conducted, we hope these data not only answer the question of statistical significance, but of clinical relevance as well. As current CEP strategies are unlikely to decrease a clinically relevant rate of disabling stroke in the all-comer population, it could be more important to identify patient categories that benefit from CEP due to an increased a priori stroke risk. The largest randomised trial to date did not find a significant treatment modification effect for any subgroup,23 but patients with a history of stroke and peripheral arterial disease were more likely to benefit from CEP in the STS-TVT registry.4 Despite the fact that the STS-TVT-based prediction model only had a modest predictive value for stroke—with a C-statistic of 0.6229—the evaluation of such modification effects could ideally be assessed in a patient-level pooled analysis of the included trials in this study.

Current early-generation devices do not seem to pass the test of clinical relevance, but they may still provide a path forward for next-generation CEP devices. Several new devices have been introduced, with encouraging results. These are designed for greater debris capture and overcome many of the limitations inherent to current systems, such as incomplete arch vessel coverage and interaction with the TAVI delivery system.

Limitations

The present meta-analysis incorporates all studies evaluating early-generation CEP devices and whether or not they have been deemed effective. Therefore, we included both shield-based and filter-based CEP devices. Although there is a lack of direct comparative studies assessing potential differences in effectiveness between these two strategies, they conceptually differ. All three included shield-based CEP studies (DEFLECT III, REFLECT I and II17 21 22) did not observe a beneficial effect of CEP on the outcome of (disabling) stroke. We can only hypothesise on the underlying cause of these findings, but this may be related to the suboptimal device success rate, which was reported to be only 57.3% in REFLECT I, as assessed by an independent angiographic core laboratory.21 Subgroup analyses of three studies may be less meaningful, but a recent similar meta-analysis under the frequentist framework found a beneficial effect of CEP on disabling stroke in patients treated with the Sentinel device.30 However, this effect seems to be mainly driven by the PROTECTED-TAVR trial.23

Based on this apparent effect in PROTECTED-TAVR, and the important implications on prognosis and quality of life, disabling stroke was elicited as the primary outcome of the current analysis. However, also minor stroke and even silent stroke have been related to a decline in cognitive function and a reduced quality of life,11 and should be weighed as well.

Several aspects of Bayesian inference can be considered subjective, including the selection of the prior and the formulation of the MCID. Therefore, our principal meta-analysis was conducted under a vague prior (assuming no difference, with a weakly informative distribution). Furthermore, to ensure a fair and transparent assessment, we elicited two different informed literature-based priors. These informed priors were derived from a large registry and national database that undertook several efforts to reduce bias, including matching strategies.4 16 Then, the MCID was based on published external expert consensus13 instead of our own belief. Still, when we would have applied less conservative MCIDs derived from earlier published consensus, the posterior probability of a clinically relevant treatment effect would only decreased further.

Conclusion

This Bayesian meta-analysis of randomised trials on the effect of current CEP devices during TAVI on disabling stroke as a primary outcome found a high probability of a beneficial CEP treatment effect. However, it was deemed improbable that this was a clinically relevant reduction in disabling stroke rate. As such, future research should focus on the development of new-generation CEP devices, and the identification of TAVI patients with an increased baseline risk of postoperative stroke, to achieve a clinically relevant reduction of the disabling stroke rate.