Evaluations quantify the link between a given intervention and its intended outcomes. They confirm and strengthen the evidence of intervention efficacy or effectiveness and can reveal heterogeneity of outcomes. These evaluations are one of the primary forms of evidence used to influence policy, practice, and funding decisions for foundations, non-governmental organizations, governments, and multilateral agencies. Internal and external validity of evaluations determine their value; however, external validity is hard to assess and not always addressed. One key aspect of external validity for field trials is a clear understanding of the underlying context in which the intervention was delivered and what intervention was actually delivered, including implementation fidelity (Green and Glasgow, 2006).

WASH intervention evaluations in particular present several challenges related to comparability, replicability, and generalizability. WASH interventions are complex and multi-faceted, which has led to challenges in adaptation and scale (Haque and Freeman, 2021). Variation in implementation and context drive heterogeneity in reported effects of WASH interventions. (Crider et al., 2023; Garn et al., 2017). Cost-effectiveness varies dramatically even for the same intervention, and determinants of costs and outcomes are often not well measured or reported (Crocker et al., 2017b, 2021; Venkataramanan et al., 2018).As such, evaluations should provide external validity, by revealing variability in effectiveness alongside thorough documentation of implementation, contextual considerations, and sub-population analysis.

Replicability is an important scientific concept related to evidence generation, but unlike in a controlled laboratory setting, replication of field evaluations is extremely difficult. Context and implementation influence implementation outcomes and intervention effectiveness, which leads to variation rather than replication of trial results (Crocker et al., 2021; Cumming et al., 2019; Haque and Freeman, 2021; Proctor et al., 2011; Venkataramanan et al., 2018). Several systematic reviews have attempted to derive lessons on what works in the WASH sector. As an example, Wolf et al. assessed the effectiveness of WASH interventions in reducing diarrhea, but found high heterogeneity of outcomes, perhaps due to the high variability of the interventions included (Wolf et al., 2023). Garn et al. found that inconsistent reporting of sanitation interventions made overarching comparisons of even similar intervention strategies difficult (Garn et al., 2017). An evidence mapping approach conducted by Chirgwin et al. similarly found the sector limited by the quality of reporting (Chirgwin et al., 2021). Accurate and consistent reporting of context and implementation details would facilitate comparisons of interventions effectiveness and impact, adaptation and scale, and cost-benefit analysis of public health investment strategies. Failure to report these details can lead to an inaccurate conclusion that an intervention is ineffective, when it may have been inadequately implemented (known as Type III error). (Basch et al., 1985).

Despite extensive literature on the importance and application of process evaluation and implementation science, (Green and Glasgow, 2006; Hoffmann et al., 2014; Proctor et al., 2013) robust reporting of implementation details is inconsistent in environmental health interventions, and specifically in WASH evaluations. This presents a critical evidence gap, as it limits accurate interpretation of evaluation findings, and the use those evaluations for evidence-based decision-making. This limitation is important not only for large-scale and multi-arm impact evaluations where the outcome is health, but also programmatic pre-post evaluations where WASH outcomes are the key indicator of success.

The purpose of this study was to qualify and describe the state of implementation reporting in the WASH sector. We undertook a rapid scoping review to characterize the state of reporting of WASH context and implementation in the most influential peer-reviewed WASH evaluations. We identified and characterized current reporting approaches and highlighted the challenges and shortfalls of WASH reporting. Our goal was to highlight strengths and limitations of peer-reviewed literature in particular by means of identifying improvements that could be made to the quality of WASH implementation reporting.