The Essential Newborn Care Corps (ENCC) intervention was aimed at increasing facility attendance for delivery, antenatal and postnatal care services through the recruitment, training and rebranding of TBAs to work as Maternal Newborn Health Promoters (MNHPs). In their new roles, MNHPs provide health counseling and referrals to pregnant women and new mothers. ENCC was implemented in Bo District of Sierra Leone between March 2014 and September 2016. The intervention covers the catchment areas of 18 primary health facilities, known as peripheral health units (PHUs), with a total population of about 105,000.

At the beginning of the project, 200 TBAs received a 2-week training on the promotion of birth preparedness, complication readiness and newborn care. Newly branded as MNHPs with ID cards, shirts and pictorial counseling cards, they conducted home visits. During these home visits, they gave health promotion messages, checked for maternal and newborn complications and provided referrals to health facilities for ANC, PNC, delivery, maternal and newborn complications and family planning. Additional project components included monthly meetings and supportive supervision for MNHPs, facilitated by project staff and health facility staff, as well as quarterly review meetings with project stakeholders.

The project also sought to explore how providing MNHPs the opportunity to sell health and baby products as a source of income might incentivize them in their new role and further contribute to positive health outcomes. Half of the MNHPs were enrolled in a small social enterprise. They received a 5-day business training and were provided with a no-interest loan in the form of a product basket valuing approximately US$30. They sold health and baby products to women during their home visits, made loan repayments and purchased products to build their businesses during monthly meetings. The products were sourced from wholesalers in Sierra Leone’s capital city and sold to MNHPs at a 7% profit margin. The convenience of bringing products to the doorsteps of women living in rural areas, and the affordable pricing of the products was to contribute to the viability of the business model.

To assess the impact of the intervention on target MNCH outcomes and the extent to which the business model added value, the evaluation used a three-arm, quasi-experimental design that isolated as best as possible the effects of the intervention from broader secular or contextual changes. The three arms were defined as follows:

Health promotion (HP) only arm: 100 rebranded TBAs in the catchment areas of nine PHUs with a population of about 57,040.

Health promotion plus business (HP+) arm: 100 rebranded TBAs in the catchment areas of nine PHUs with a population estimated at 46,355.

Comparison arm: Catchment areas of nine PHUs with a population of nearly 54,700.

Baseline and endline household surveys were conducted with women age 15–49 who had a live birth in the year prior to the survey, or who were pregnant at the time of the survey. The surveys were administered in October–December 2013 and June–July 2016. In each survey, a two-stage sampling procedure was utilized, with enumeration areas (EAs) selected in the first stage using probability proportional to size (PPS). In the second stage, a complete listing of households was carried out in each selected EA, after which a number of households were systematically selected using a sampling interval determined from the total number of households in the EA and the sample of households needed for each EA. The baseline questionnaires were drawn largely from the 2008 Sierra Leone demographic and health surveys (DHS) and adapted to the context of Bo District. The endline tools were similar to the baseline tools, and included variables on the exposure to the intervention. The final sample included 795 eligible women from 66 EAs at baseline, and 1110 women from 92 EAs at endline.

In this paper, we examine the impact of the intervention on its six primary health outcomes: (1) Initiation of ANC during the first trimester of pregnancy, (2) Four or more ANC visits during pregnancy, (3) Health facility delivery, (4) PNC for mothers by a health professional, (5) PNC for newborns by a health professional and (6) Initiation of breastfeeding within 1 h of delivery. The baseline and endline surveys also collected data on three other MNCH-related outcomes which are part of the variables required to estimate the lives saved: (1) Tetanus toxoid vaccination during pregnancy, (2) Iron supplementation during pregnancy and (3) Preventive treatment of malaria during pregnancy.

This study focuses on women who had a child in the year preceding the surveys. In both the baseline and endline surveys, the majority of the study population had no education and were predominantly Muslim. Just over half of the women were between the ages of 20 and 29 years. The analyses control for the following variables: household wealth and women’s education, age and religion; the distributions of which are shown in Table 1.

The cost analysis of the ENCC project followed a narrow health sector perspective, covering both the financial costs (e.g., cash outlays to support the functioning of the project), and the economic costs (e.g., staff time not paid for by the project, MNHPs time and transport). An “ingredients approach” was utilized for the quantification and valuation of inputs (Drummond et al. 2015). These costs are specific to the Sierra Leone context. We did not include valuations of societal costs, such as the opportunity costs of investing in alternative programs (e.g., the benefits measured in life years saved of investing in the next best alternative health program), the time and travel costs of patients and family, and valuations of societal benefits (e.g., the value of productivity gains from mortality averted and the benefits to patients and family outside of the health gains). This choice was made largely for reasons related to the difficulty of identifying and quantifying such costs and benefits. Capital costs (e.g., equipment and vehicles) were assigned resale prices. Each cost item was apportioned to the two intervention arms (HP and HP+) in proportion to the contribution of each arm to the cost or based on estimated time committed to the different arms. An exchange rate of US$ 1 = 4050 Le (the local currency) was used. The rate ranged from 3750 to 4350 during the project period.

Cost estimates were derived from the project’s financial system, with the exception of opportunity costs, whose estimates were derived as follows:

Maternal Child Health (MCH) Aide Opportunity Costs: One MCH Aide at each of the 18 target PHUs was estimated to spend 3 days per month on supervision visits, meetings and handling of referral forms at PHUs. The salary used was US$ 55 per month, as reported in another study (Frontline Health Workers Coalition 2015).

MNHP Opportunity Costs: Project reports suggest each MNHP spent about 9 days per month conducting home visits, accompanying women to PHUs for delivery, and participating in monthly meetings. According to the World Bank, Sierra Leone’s GNI per capita in current US$ was $420 in 2010 and $490 in 2016, for an average value of $478.30 during the project period (interpreted as the annual earning for an average Sierra Leonean worker). Estimated MNHP earnings were discounted by 40% to account for the fact that MNHPs are unskilled workers. Each MNHP received a non-cash incentive valuing 5000 Le during the monthly meetings.

MNHP Transportation Not Covered by the Project: Project monitoring data shows that each MNHP made an average of 10 referrals per month, a third of which were for delivery. Generally, women were accompanied by MNHPs for delivery. The cost incurred by MNHPs to accompany women to the health facility was zero when the PHU was within walking distance and ranged from 500 to 5000 Le (one way) when another form of transportation was required. An estimated median value of 1500 Le was used for this evaluation.

Difference-in-differences (DID) regression models (Heckman 2005; Bertrand et al. 2004) were used to quantify the impact of the intervention on the target outcomes, according to following Eq. (1):

$$ }\left( }} }}} \right) = \beta_ + \beta_ A_ + \beta_ P_ + \beta_ A_ P_ + X\emptyset + \varepsilon_ . $$

(1)

where Pijt = Pr (Yijt = 1); Yijt is an outcome for a woman i (i = 1, 2, … Nj), from enumeration area (EA) j (j = 1, 2, … Mt) at time t (t = 0, 1). Aj represents the study arm (indexed on the EAs), taking the values 1, 2 and 3 for the comparison arm, HP only arm and HP+ arm, respectively. Pt is a dummy variable coded as 0 for baseline and 1 for endline; X is a vector of individual level covariates (household wealth and respondent’s education, age and religion in this case); β0, β1, β2, β3 and \(\varnothing \) are the corresponding regression coefficients; and εijt is the error term clustered by EA. The DID estimate of interest is the coefficient β3 of the interaction between the variables Aj and Pt.

Four versions of the above model were run, comparing each of the two intervention arms with the comparison group, comparing the two intervention groups with each another, and comparing the combined intervention groups with the comparison arm. STATA 14 software was used for the analysis.

In the cost-effectiveness analysis, we use standard guidelines for economic evaluations of health interventions (Drummond et al. 2015), including the use of the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist (Husereau et al 2013). To estimate the number of maternal and child lives saved from improvements in the coverage of the outcomes of interest, we use the Lives Saved Tool (LiST) of Spectrum (version 5.47). LiST is a modeling software developed to estimate the impact of scaling up health and nutrition interventions for maternal, newborn and child health (Walker et al. 2013; Winfrey et al. 2011). The LiST model requires baseline and endline prevalence estimates of the outcomes and makes use of the built-in country-specific demographic data (Sierra Leone in this case). These indicators were generated for each arm using predicted probabilities from the logistic regression model in Eq. (1) above. The LiST model for each study arm was run, and national-level lives saved according to the population of the study arm were prorated and generated. The lives saved were further converted into discounted life years saved, making a number of assumptions including a 5% discount rate (to account for future benefits being weighed differently than present benefits), an average age of child death of 1 month, and a life expectancy of 50 years. As a result, each death averted was associated with approximately 49.9 years of life gained, which discounted at a 5% rate, amounted to 19.26 discounted years of life gained for every death averted.

The measure of cost-effectiveness, the incremental cost-effectiveness ratio (ICER), is defined as (CI-CC)/(MI-MC), where CI and CC are the total costs related to the intervention and comparison groups, respectively; and MI-MC are the life years saved in the intervention and comparison groups, respectively. Because the lives saved are incremental, CC = 0. ICER thus represents the average incremental cost associated with one additional life year saved. Our interpretation of cost-effectiveness is based on the World Health Organization (WHO)’s recommendations (WHO 2014). Interventions are classified as highly cost-effective if ICER is less than the country’s GDP per capita (US$638.3 for Sierra Leone), and cost-effective if ICER is less than three times the country’s GDP per capita (US$1915 for Sierra Leone).