We welcome Knight & Cook insightful comment on our review. 1 Incomparability of treatment groups is often blindly ignored in randomised controlled trials – a post hoc analysis of baseline characteristic tables". Clearly, a prognosis-blind standardised mean difference is no perfect measure of baseline incomparability in trials. An optimal measure should quantify the impact of covariate imbalance on the estimation error, by taking into account information on covariate prognostic strength. 2 Comparability of Randomised Groups. The reason may be formulated following the framework of omitted variable bias. 3 Mostly harmless econometrics: an empiricist's companion. For simplicity, suppose an outcome that depends on a linear combination of a baseline covariate X and the treatment assignment A (by convention, A=1 denotes the ‘treated’ and A=0 the ‘control’ arm). One can show the naïve mean difference in outcome across the treatment arms, noted β^, is equal to βA+βXαX, where βA denotes the true treatment effect to be estimated in the sample, βX the covariate prognostic strength, and αX the mean difference in X across the treatment arms (A=1 and A=0). Note that αX is an expression of covariate imbalance on a natural, non-standardised scale. Thus, the gap between the truth and the estimate (i.e. estimation error), β^−βA=βXαX, is merely the product of the covariate prognostic strength times the (non-standardised) covariate imbalance. Similar products have been proposed as prognosis-weighted imbalance metrics in the propensity score literature. 4 Belitser SV Martens EP Pestman WR et al. Measuring balance and model selection in propensity score methods. , 5 Caruana E Chevret S Resche-Rigon M et al. A new weighted balance measure helped to select the variables to be included in a propensity score model. , 6 Nguyen TL Collins GS Spence J et al. Comparison of the ability of double-robust estimators to correct bias in propensity score matching analysis. A Monte Carlo simulation study. Figure 1 depicts the influence of αX (i.e. imbalance) and βX (i.e. prognostic strength) on the estimation error, following the above formula.

Figure 1Theoretical estimation error in trial results according to covariate imbalance and covariate prognostic strength, under a linear additive model. A systematic review assessing imbalance without quantifying prognostic strength (e.g. our study) may be regarded as a greyscale-version of this figure. (In this illustration, covariate imbalance follows a standard Normal distribution, and covariate prognostic strength a Uniform distribution between -1 and 1.)