Calculation and interpretation of an E value

When an outcome is predicted using a risk ratio (RR), the E value is calculated as:

$$}=}+\surd }\times (}-1)]}$$

For outcomes predicted using odds ratios and hazard ratios the same equation is used, although some adjustments are made, based on how common the outcome is (e.g. >15% of individuals have an outcome at the end of follow-up).

Specific E values should not be considered as generically ‘likely’ vs. ‘unlikely’ evidence of confounding potentially explaining exposure-outcome observations, as inference should be based on a case-by-case basis. For example, Gaster et al. [7] conducted a meta-analysis on the association between anti-depressant use and miscarriage risk, concluding that risk of miscarriage was higher among anti-depressant users (RR = 1.41). For this RR, the E value = 2.17. Alcohol use was considered as a potential confounder because pregnant women who use anti-depressants are at much higher risk of excessive alcohol consumption than pregnant women who do not. The authors went on to conclude that alcohol use could explain the association between anti-depressant use and miscarriage risk because the relationship (expressed as a risk ratio) between anti-depressant use and alcohol, and alcohol and miscarriage risk are both known to be >2.17. Yet, it is important to note that the strengths of the association between confounder and exposure and confounder and outcome do not both need to exceed an E-value to provide statistical evidence of potential ‘total’ confounding.

Used alongside the E-value, is the joint bounding factor, B:

RRUD is the size of association between the potential confounder and outcome and RREU is the size of association between the potential confounder and exposure. To explore how combinations of confounder exposure and outcome relationships could combine to create statistical conditions for ‘total’ confounding, one sets B (bounding factor) to the E value. In simple language, if the likely size of association between the potential confounder and outcome (or exposure) is larger than E value but the potential confounder and exposure (or out outcome) association is smaller than the E value, the two may still combine to be equal to or exceed the E value and therefore contribute to ‘total’ confounding. For instance, if an E value is 2.00 (RR ~ 1.33), a stronger association between confounder and outcome (RR = 3.00) and a weaker association between confounder and exposure could exist (RRs > 1.60) to produce ‘total’ confounding and explain away the effect (see Fig. 1).

Fig. 1: Example visualisation of the bounding factor.

Red dotted lines show a hypothetical example combination of associations between confounder and outcome (RRUD) and confounder and exposure (RRUE) that would explain away an effect of RR = 1.33. Number in brackets is E Value [RR = 2] and denoted by a black circle on the curve.

This equation can also be used to estimate by how much (% of effect) combinations of known RRUD and RREU could account for an observed effect (RR). B = the observed RR, would indicate combinations of RRUD and RREU could feasibly create ‘total’ confounding and explain away the observed effect. B = 50% of observed RR, would indicate that combinations of RRUD and RREU could feasibly explain half of the size of the observed effect (‘partial’ confounding).

UPF consumption and weight gain: identifying potential confounders

Higher UPF consumption has been identified as a potential causal contributor to weight gain and obesity largely on the basis of observational research. Because both UPF consumption and obesity have sociodemographic and personal characteristic patterning, variables such as age, gender, social class, physical activity and smoking status are typical control variables in study analyses [2, 8] due to concerns over potential confounding. However, other potential confounders could be ‘unmeasured’ and therefore not statistically controlled for in any given study. In the present analyses we consider a person’s trait tendency to overeat and experience depression symptoms, as two examples of ‘unmeasured confounders’.

Socioeconomic status (SES) is a particularly important control variable in diet and health studies. Statistically controlling for standard SES measures like education level may not fully capture the various ways by which social class could indirectly contribute to both UPF consumption and weight gain, resulting in ‘residual’ confounding [9]. Residual confounding (confounding that remains after statistically controlling for a variable) is typically very difficult to measure, but could be relevant to diet because low SES greatly increases likelihood of decreased access to healthy nutritious food, also known as food insecurity [10]. SES measures are somewhat associated with food insecurity [10], but unlikely to capture the negative consequences of food insecurity, resulting in residual risk. Here we therefore treat food insecurity as a quantifiable example of ‘residual’ confounding.

Analyses

To provide an effect estimate of the association between UPF consumption and weight gain, we identified prospective studies examining UPF consumption and risk of weight gain among adults from two recent systematic and one recent narrative review on the topic [1, 11, 12]. Five prospective studies were identified and meta-analysed. We focused on study effect estimates from analyses relating to weight gain from baseline. See Table 1. We extracted results from models that allowed for prospective weight gain to be accurately quantified and adjusted for the most comprehensive collection of potential confounders, which included demographics (social class, age, sex) and personal characteristics (physical activity, sleep, smoking status) across studies.

Table 1 Prospective studies included in meta-analysis.

Random effects meta-analysis using a Restricted Maximum Likelihood estimator were conducted using the ‘metafor’ package in R. Hazard Ratios were converted to Risk Ratios using the ‘toRR’ function from the ‘EValue’ package. We conducted separate meta-analyses to determine the effects of daily UPF intake on weight gain outcomes across different quartiles of UPF consumption (see Table 1), with quartile 1 (lowest consumption of UPF) used as a comparator in each meta-analysis. For the purpose of E value analysis we focused on Q1 vs. Q4 effect estimates, as this constituted the largest association between UPF consumption and risk of weight gain.

E values were calculated using the ‘evalues.RR’ function from the EValue package (see Table 2). To characterise size of associations potential confounding variables have with UPF consumption and risk of weight gain, we drew on available published meta-analyses and high quality large epidemiology studies. To convert Odds Ratios to Risk Ratios for the confounding effects we used the formula RR = OR/(1 – p0 + (p0 * OR)) where p0 is the baseline risk [6]. To do this we used the ‘ORToRelRisk’ function from the ‘DescTools’ package [13]. We used baseline risk estimates from relevant studies or conservative estimates if not available directly. Data and R code can be found here https://osf.io/z89pa/.

Table 2 Meta-analysis of UPF consumption and risk of weight gain studies.

Table 2 presents the meta-analysis estimates. Figure 2 illustrates the RRUD and RREU values that equate to E values. Meta-analysis revealed statistically significant and small associations between UPF consumption and weight gain outcomes, based on effect size interpretation guidance [14]. For the largest effect estimate (UPF quartile 4 vs. 1), E values for the point estimate and its lower confidence interval = 1.55 and 1.43, respectively, indicating that unmeasured confounders associated with both UPF consumption and weight gain to a similar degree (small to medium effect sizes) could nullify associations.

Fig. 2: Visualisation of the Bounding Factor for UPF consumption and risk of weight gain.

Red line is RR association of trait overeating and risk of weight gain, and green line represents the strength of RR association between trait overeating and UPF consumption needed to fully explain the association between UPF consumption and weight gain. Number in brackets is E Value [1.55] and denoted by a black circle on the curve.

Potential unmeasured confounding

No studies controlled for mental health or depression symptoms as potential confounders. Previously published meta-analyses indicates that higher depression symptoms are associated with UPF consumption [15] (OR = 1.44 ~ RR = 1.39) and predict development of obesity [16] (OR = 1.58 ~RR = 1.48). The joint bounding factor (RRUD * RREU / (RRUD + RREU − 1)) = 1.10, suggesting that depressive symptoms could account for 71% (effect estimate) or as much as 100% (lower confidence interval of effect estimate) of the meta-analysed association between the highest vs. lowest UPF consumption groups and weight gain.

No studies controlled for eating traits, such as overeating. Tendency to overeat has a genetic basis and is typically characterised as either general disinhibited overeating or emotional-based overeating. Both are associated with risk of higher BMI across multiple previously published meta-analyses and effect sizes are medium in size [17, 18]. For instance, the pooled association between disinhibited overeating and BMI is r = 0.28 (~OR = 2.88 [95% CI: 2.02–4.44] ~RR = 1.49 [95% CI: 1.34–1.63). Prospective studies of the association between tendency to overeat and weight gain produce similar estimates [19, 20]. A positive relationship between tendency to overeat and higher UPF consumption would seem plausible, but there is a lack of data to confidently estimate effect size with precision and we therefore concluded E value calculation was not feasible. However, as denoted in Fig. 2 an RR = 1.62 (small to medium in size) would be needed to fully explain the meta-analysed association between the highest vs. lowest UPF consumption groups and weight gain.

Potential residual confounding

All studies controlled for SES indicators, but not access to healthy nutritious food or food insecurity specifically. Food insecurity is associated with higher UPF consumption [21] and a recent epidemiological survey study estimates participants with the highest UPF consumption have a 60% higher prevalence of food insecurity (RR = 1.60) [22]. Food insecurity is associated with elevated obesity risk in a previously published meta-analysis (OR = 1.53 ~RR 1.42) [23] and effects appear similar when examined prospectively in studies [24]. Taken together, it suggests that residual confounding of this nature could account for ~86% of the effect estimate, or 100% based on the lower bound confidence interval.

Additional analyses

In an additional meta-analysis we included only studies with the same outcome (risk of developing overweight/obesity, n = 3). Results were similar to the main meta-analysis (see Table 2), though the effect estimate and E-value was slightly smaller, suggesting that confounding would be more plausible for this sub-set of studies. Corresponding Q1 vs. Q4 E value = 1.50 (Lower bound CI = 1.45).