Cost-Effectiveness of Alternative Models of Community Health Workers for Promotion of Maternal, Newborn and Child Health in Northern Nigeria

Introduction

Throughout Africa, there are critical shortages in the primary health care workforce, despite its importance in achieving the Sustainable Development Goals (SDGs) [1, 2]. Faced with declining oil prices and challenges of non-diversification of products for internally generated revenue, Nigeria is experiencing threats to its ability to provide primary health care services, particularly in the poorer states in the northern part of the country. In 2013, nationwide less than half (46.5%) of pregnant women in rural areas received any antenatal care services (ANC), and less than one-fourth (22.7%) of deliveries were taken by skilled birth attendants at the primary health centers in the country. In the northern states, the proportion of births at health centers is even lower at 10-16%, with about 1 in 5 births taking place at home with no one present [3, 4].

Nurses and midwives alone cannot fill the gap in the provision of maternal health services in the rural areas of Nigeria. First, there is a shortage of trained medical providers. Nigeria’s public health sector employs only an estimated 13 doctors and 92 nurses/midwives per 100,000 population [5, 6]. There are simply not enough nurses and midwives to meet the minimal requirements of two midwives per shift in primary health centers [7]. Second, even if there were enough doctors and nurses, they are reluctant to be stationed in rural areas, where the attrition rate is two times higher for nurses and three times higher for doctors posted to rural areas, compared to urban areas [5]. Third, with the economic downturn, neither state nor local governments can afford to supply all primary health care needs with nurses or midwives. The remunerations for midwives and nurses are significantly higher than those of community health workers (CHWs), so it is preferable to invest in more CHWs, who are also less prone to relocating away from the rural areas [8]. Primary health care centers now aim to recruit more CHWs to meet the health workforce needs of communities [7]. Finally, and perhaps of greatest concern, motivating and linking women to their local primary health care services can best be done by women working in the community. To fully meet the SDGs for maternal health, female CHWs based in the community are needed. These CHWs can provide basic ANC services in the community and then link the women to care at the clinic for complete antenatal care and skilled birth attendance [9].

Many of the evidence-based interventions for the reduction of maternal, neonatal and child morbidity and mortality can be delivered effectively by CHWs. This is shown by programmes in Ethiopia, Indonesia, India, Bangladeshi, Nepal, Zambia, and Uganda, among other low and middle-income countries [10-14]. If fully deployed throughout rural areas of low and moderate income countries, it is estimated that CHWs can contribute to the prevention of millions of infant and child deaths per year [10]. Furthermore, CHWs have proven successful in mobilizing rural populations to utilize primary health services where skilled health workers can support them [12]. Recruiting and training women to become CHWs in rural areas would yield additional societal benefits, such as empowerment of females and increased income for households with paid CHWs [14].

Following the recommendations of the One Million CHW Task Force [12], Nigeria began to develop a road map for deploying CHWs to rural communities [9]. The primary category of CHWs found throughout the Nigerian health system is the Community Health Extension Worker (CHEW or Junior CHEW, the JCHEW), who pursues a 2 to 3 year course at a school of health technology and then is employed as a primary health care provider in a rural primary health care post. In 2012 Nigeria had over 62,700 registered community health (CHWs) workers serving a national population of about 150 million. This gives a ratio of less than 1:2000 population, well below the standard of 1: 650 inhabitants, as recommended by the task force. Furthermore, due to shortages of nurses, midwives and doctors, these CHWs (CHWs, CHEWs and JCHEWs) have only been stationed at health facilities, rarely working within the community itself. A swift and significant increase of CHWs across Nigeria therefore requires the establishment of a new cadre of community health workers, to be based in the rural communities where they work directly with residents who otherwise would have no access to primary care or any outreach services.

There are many different models for CHWs, so before rolling out a new national CHW initiative, Nigeria’s National Primary Health Care Development Agency wanted to determine which model would best fit the country’s needs. They wanted to base decisions upon the relative differences in cost-effectiveness between possible CHW models under consideration. Assessing the costeffectiveness of CHWs has been a challenge, given their mix of voluntarism, differences in integration with the formal health care system, and often rudimentary systems for tracking their activities [15, 16]. However, there is now an emerging literature which documents the cost-effectiveness of CHWs to promote diverse primary health outcomes, including screening and treatment of tuberculosis, family planning method utilization, improved child health care practices including better nutrition and sanitation, use of bed nets to prevent malaria, and management of fever in newborns and infants [10, 17-24].

Useful as these studies are, cost-effectiveness analyses need to be tailored to the very specific differences in training and responsibilities assigned to the CHW, as this influences both the costs per CHW and the relative differences in the CHW impact on health outcomes. In Ghana, for example, higher cost communitybased health nurses contributed to reducing child mortality, while the lower cost community volunteers did not [25]. Northern Nigeria has had multiple CHW models operating, including some based primarily in the community. These models range from the CHEW who has completed the certificate training through to a diverse array of community volunteers (CVs).

The CHEWs posted to the community are close to the Health Extension Workers implemented by Ethiopia, upon which the One Million CHW Taskforce initiative is based. The CHEWs have completed their certificate program, are directly hired by the health care system, and then receive additional training and payments to provide community-based service delivery for women living in the most distant hard-to-reach communities included in the clinic’s service catchment area. The Millenium Villages and several large non-governmental organizations have piloted variants of this model in Nigeria.

At the other end of the CHW continuum are the CVs, who are members of the communities they serve, selected by the community leadership based on their willingness and trustworthiness and who then receive little more than a one-day orientation before being asked to mobilize parents and caretakers for a vaccination campaign, distribution of bednets, or other community-wide prevention activities. CVs typically have been mobilized only on an occasional basis, but CVs may also be part of a longterm strategy by non-governmental organizations to inform and mobilize community members on a wider set of maternal and child health issues, such identification of malnourished infants and children to be referred for nutritional support programs.

In between these two extremes of CHWs are a mid-range group who are still considered volunteers, recruited from and working in the community, but who receive more training and a small stipend as an incentive for their work. Under the leadership of the Subsidy Reinvestment and Empowerment Programme on Maternal and Child Health (SURE-P MCH) initiative, Nigeria has recently rolled out a cadre of Village Health Workers (VHWs) who occupy this middle niche. Focusing on the promotion of maternal and child health, the SURE- P VHWs are young women who are recruited from and work within the community, but they are integrated into the primary health care system at the local clinic, where they work in close collaboration to support midwives and CHEWs providing maternal and child health services to the women they referred to the clinic [9].

Therefore, before proceeding with further scaling up of CHW models in Nigeria, a cost-effectiveness analysis was undertaken to assess the relative and absolute differences in each model [26]. This study reports on the cost-effectiveness analysis comparing three CHW models operating in Northern Nigeria in the three states of Jigawa, Katsina and Zamfara:

1. Community Volunteers (CV): Recruited from their home village, minimally trained, and responsible for community mobilization and group education on maternal and child health topics.

2. Village Health Workers (VHW): Recruited from their home village, and trained to provide basic health education on maternal health topics, and to link women to services at the closest PHC, with which the VHW are affiliated.

3. Junior CHEW in the community (JCHEW-CBSD): Previously trained JCHEWs who receive additional training in order to provide basic MCH services in villages within a specified catchment area.

Methods

Description of each CHW Model

Community Volunteers (CVs) Model

In Northern Nigeria, the most extensive implementation of the CV model was by the Partnership for Reviving Routine Immunization in Northern Nigeria (PRRINN), as part of its maternal and child health extension (PRRINN-MNCH). With support from the UK Department of International Development and the Norwegian Government, from 2009-2014 the partners worked with the government and non-governmental organizations in the four states of Jigawa, Katsina, Yobe and Zamfara to strengthen the delivery of primary health care services, particularly in rural areas. The systems improvements introduced by the partnership used a cluster approach, developing a tiered system of maternal and emergency obstetrical care facilities so that women throughout the program area would have access to basic or complete maternal health services at either their primary health care center or the referral hospital. Prior to 2012, all clinics encompassed by the PRRINN-MNCH partnership had received support for improved midwifery and primary care services. Complementing these clinical improvements, the program recruited male and female volunteers in each community who were trained to lead community discussion groups to promote greater awareness of MNCH care, as well as to help establish emergency transport and emergency blood donors for women to use to go to the clinics for their deliveries. Each Ward Development Committee nominated 30 men and women aged 18-45 years to be community volunteers (CVs). (For details on their recruitment, training and supervision see Table 1.)

The CV’s were provided a short training in using a participatory teaching method, “See, Say, Do,” which teaches non-literate participants to demonstrate their understanding of the key message through songs and rhymes, many accompanied by appropriate body motions. They encouraged women to go to health facilities for skilled birth delivery and helped establish community emergency transport services. The CVs were expected to spend a few days per month organizing community discussions, meetings with groups and following up with pregnant women. Their target was to engage about 30% of the community in 12 group educational sessions, each with 15-20 participants. At the end of the five-year program, the program partners had recruited and trained 30,840 community volunteers in 830 communities in 73 LGAs. Most of the CVs volunteered for a period of 1-3 years. They received no stipends of incentives but received per diems for the travel and time spent in annual refresher training or for monthly supervisory meetings with the PRRINN-MNCH local engagement coordinators. Because of its extensive implementation and the indication of its impact, the PRRINN-MNCH CV program was selected to represent the CV model in these analyses. At the time of this assessment there were an average of 1727 active community volunteers per state.

Village Health Workers (VHWs) Model

Since 2009, the Nigerian government has been pursuing a series of strategies aimed at strengthening service delivery for its underserved rural population, particularly women and children, including the Midwives Service Scheme to encourage more women to work as midwives in rural areas. Experience with that initiative showed that providing midwives was not enough, and a complementary cadre of CHWs was needed to engage and mobilize women to seek maternal and child health care at the facilities where midwives had been posted [8]. In 2013 the Nigerian Subsidy Reinvestment and Empowerment Programme, Maternal and Child Health Programme (SURE-P MCH) launched the SURE-P MCH Village Health Worker program. At selected clinics in underserved rural areas where a midwife was already posted, two clinicbased CHEWs were trained to mentor 3 Village Health Workers (VHWs). VHWs were recruited by the Ward Development Committee according to the eligibility criteria provided by the SURE-P MCH programme: female, age 25 and above, married, and preferably literate. Most of the VHWs recruited already had experience serving the community, previously working as community volunteers mobilizing families during immunization campaigns or as traditional birth attendants (TBAs). The VHWs received a small monthly stipend from the SURE-P MCH fund, and they were expected to visit an average of 30 families per month. They were expected to meet weekly with their CHEW mentors at the clinic to discuss their activities. By 2013, there were 96 SURE-P MCH VHWs per state, allocated 6 per participating primary health care clinic (See Table 1).

The VHWs received a 6-day training during which they learned how to make home visits and give educational talks, using the Pictorial Guide to explain the recommended behaviors. They were asked to start by identifying pregnant women and then to make home visits to these women, as well as to mothers with newborns or young infants. During these home visits, they provided information on the key recommended maternal and child health promotion practices, engaged small groups of women for group education, and encouraged or directly assisted women to go to the health facility for their antenatal care visits or to have their infants delivered by skilled birth attendants. The VHWs kept records of visits using a simple pictorial tally sheet.

JCHEW-CBSD Model-Although the bulk of the PRRIN-MNCH community engagement focus was on the nurturing of a large group of community volunteers to encourage women to use the maternal and child health services, there remained great unmet need for services in the most hard-to-reach communities. Building on the work of the One Million CHW Task Force, the PRRINNMNCH team worked with its state and local government partners to establish a post for a JCHEW charged with community-based service delivery (CBSD). In 2012-13, PRRINN-MNCH recruited JCHEWs graduated from their state’s Health Technology Institute and gave them an additional 2-week training on communitybased service delivery, using the UNICEF-WHO Integrated Management of Newborn and Child Illnesses (IMNCI)guidelines and job aids as a foundation. They learned how to use participatory education techniques, such as those used by CVs. In 2013, there were 50 JCHEW-CBSD in each state, both female and male, aged 26-48 years (See Table 1).

The JCHEW-CBSD providing CBSD were full-time employees of the PHC, and received a salary from the local government area authority, as well as support for their transport and travel per diems while on rotation. In total, they covered an average of 145 communities per state. They were supervised by the PRRINNMNCH local engagement coordinators, as well as the MCH focal point for the Local Government Area (LGA). In contrast to the CVs and VHWs, the JCHEW-CBSD delivered primary care services in the communities they visited, and they referred urgent cases to the PHC or the hospital. They spent 3-4 days per week (20-28 hours per week) on rotation in the communities, spending one day per village. In addition, they spent 1-2 days per week meeting with families at the primary health care center (PHC) where they were based.

Data Sources

Study Population

The study was conducted in 2013-2014 in three northern Nigeria States of Jigawa, Katsina and Zamfara, where the PRRINN-MNCH program had been operational since 2009. These were all states in the northern regions where primary health care services and routine immunization services were being revitalized after having been suspended after anti-vaccination activities in 2005. Yobe was excluded due to program reductions related to Boko Haram activity in the state. We assessed maternal, newborn, infant and child health outcomes in the participating local government areas using both population-based household sample surveys and clinical consultation data for maternal and child health from the clinics serving the population.

By 2013-14, there were 5,181 CVs , 1727 per state, distributed across all 27 LGAs in Jigawa, all 14 LGAs in Zamfara, and across 17 LGAs, half of the 34 LGAs in Katsina. In these LGAs the CVs were concentrated in the hard to reach communities located more than 5 km from a primary health care clinic. These CVs provided support to 2.7 million persons, 21.5% of the total state population, 24.3% in Jigawa, 21.8% in Katsina, and 17.5% in Zamfara. In each state 50 JCHEW-CBSDs were deployed as a complement to the CV model in especially hard to reach areas, covering 16 of the 27 LGAs in Jigawa, 8 of the 17 LGAs in Katsina, and 12 of the 14 LGAs in Zamfara. There were a total of 150 JCHEW-CBSDs included in the study.

The 288 VHWs worked in 16 LGAs in each state where the SURE-P MNCH program had identified rural clinics where it was supporting the posting of a midwife, training of CHEWs and recruitment of VHWs. They provided for the recruitment and support of 96 VHWs per state. In the LGAs where there was overlap between the VHW and CVs, the CVs in the overlap areas were excluded from the analysis. There were no coverage overlaps between the VHW and JCHEWs.

Study Design

We used a quasi-experimental design model to assess differences in health outcomes for the comparison communities (where no CHW model was implemented) versus those from the three alternative CHW models: (1) Community Volunteer (CV) model, (2) SURE-P MCH Village health worker (VHW) model and (3) Junior Community Health Extension worker (JCHEW-CBSD) model. The comparison population for all three models were women with children under age 5 who had not been served by any of the CHW models, but who were still within the same hard-to-reach type of community. The impacts and costs were assessed as of 2013 when all three programs had become fully operational.

Effectiveness Measures

Monthly CHW activity logs: The first set of effectiveness outcomes were the 2013 monthly activity logs of the CHWs and their CHEW supervisors’ monthly reports. While not used directly in the cost-effectiveness calculations, these data provide a measure of the actual implementation of the CHW activities, a measure of the CHW interactions with community members.

Population-based sample survey of MNCH outcomes by interactions with CHWs: The second set of outcome data were based on the 2013 individual self-report of interactions with CHWs and health behaviors that the individual reported adopting pursuant to these interactions. In May-June 2013, six months after the launch of the VHW programme and two years after the launch of the other two CHW programs, PRRINNMNCH conducted a random household survey to assess changes in health behaviors and outcomes in the local government areas where PRRINN-MNCH interventions had been implemented since 2009. A stratified cluster sampling design was used to ensure inclusion of communities where each of the three CHW models had been implemented. In the 51 LGAs sampled in the three states participating in the cost-effectiveness study, 3321 women aged 15- 49 who had given birth in the previous five years responded to the survey, along with 441 in comparable control areas where none of the CHW models were implemented. The survey questionnaire included items assessing interactions with CHW by type, key MCH promoting behaviors, ANC consultations and skilled birth attendance for pregnancy in the last five years. The analyses categorized these key variables by category of CHW with whom they interacted: No CHW, CV only, VHW, JCHEW- CBSD. Significant differences in the outcome variables were assessed using the Chi-square statistic.

Monthly Clinical Consultations: The third set of data for outcomes were aggregate clinical consultation data for 2013- 2013, obtained from the PHC consultation logs at clinics serving populations covered by any of the three CHW models. These data are reported monthly by the In-Charge of every primary health clinic in Nigeria to the state and then the federal primary health care development agency. We abstracted the selected health outcome indicators from the electronic annual summaries and then aggregated them by year (2012 and 2013) and by the type of CHW providing services to that clinic.

Key effectiveness measures used across all sources were:

- Number of people reached by home visits or group sessions led by the CHWs;
- Percentage of pregnant women with any ANC visit; women with four or more ANC visits; those who took two or more steps to prepare for birth;
- Knowledge of maternal, newborn or sick child danger signs
- Births with a skilled birth attendant at a facility; newborns checked within 48 hours (as measured by the variable OPV at birth); breastfeeding within 24 hours of birth
- One -year olds with all recommended immunizations

Cost Measures: Following standard practice for costeffectiveness analysis, we included all costs associated with implementing the program, including annual operating costs for each program plus the training and amortized start-up costs for that CHW cadre [26, 30]. Developmental costs were excluded, as well as costs associated with training of the JCHEWs prior to their recruitment to become a JCHEW providing CBSD. All costs were converted from Nigerian Naira to US dollars, using the December 31, 2013 posted exchange rate of 1 USD = 160.38 Naira.

Annual or Recurrent Costs: Annual recurrent costs included annual expenditures associated with planning and review meetings, supervisory or relevant administrative personnel, CHW salaries or stipends, and any annual refresher training costs. Information on these costs was obtained in two steps: 1) interviews with programme managers to determine specific staff or resource inputs for each CHW model; 2) interviews with programme managers and financial officers to provide information on monthly salaries and other charges, e.g., per diems, transport costs, supplies for training workshops. The Federal, State, local government and PRRINN-MNCH program inputs for annual planning and review meetings regarding the CHWs were estimated based on the 2013 budgets for each programme model.

• Recurrent materials costs included all supplies and materials, including the restocking the CHW kits and re-supplying Mama Kits (infant wash basins and wraps for women delivering at facilities supported by CHWs). These items were obtained directly from the agency expenditure records.
• Routine personnel costs included any stipends or salaries for the CHWs and any supervisory personnel. Annual salary or stipend cost were allocated based on the proportion of the year allocated by the recipient to the pertinent CHW activities.
o CHW stipends: For the CVs, this included the per diem that they received while in annual training. For the VHWs, this was their annual total monthly stipends, and for the JCHEW-CBSD, this was their annual salary.
o Supervisor time costs: For the CV and JCHEW-CBSD supervisory staff, we first ascertained how many days per month they spent on supervision activities, and then applied this proportion to their monthly salary, and then converted to annual costs. For the CHEW mentors in the VHW program, we only included the annual total of monthly stipends paid to them by SURE-P MCH to directly compensate them for their supervision time.
• Planning and review expenses included meetings and non-supervisory activities involving federal, state, and LGA staff. We included only the days devoted to CHW-related activities when allocating relevant salary costs per cadre.
• Annual refresher training costs: Both the CV and JCHEW-CBSD models provided annual refresher trainings. As for the CHW and supervisory personnel expenses, we first obtained information on the time inputs of staff for providing these activities and then applied their annual salary levels to estimate the costs associated with these time inputs. The annual training costs also included the per diems provided to the CHWs and trainers while attending the training, as well as the cost of any materials reproduced for distribution to the participants.

The CVs, VHWs, and JCHEW-CBSDs were implemented under the same standard budgets at the state and/or federal level, so numbers of CHWs and their support were uniform across states. Hence, a set of harmonized costs was calculated for each model, and all states and the federal agency approved as correct for their unit. After calculating total annual costs, these were normalized to the number of CHWs active in the state at that time.

Amortized Start-up Costs: For each CHW model start-up costs included those for mobilizing the recruitment committees, training the CHWs and their supervisors, and the production of their CHW kits. Staff inputs were calculated as above, estimating the time each staff person devoted to the training or start- up activity, and then applying their salary level for that period. In the case of the VHWs, the training costs were stepped down as the state’s share of the total national training costs for the national and state trainers. Costs were calculated separately for training conducted by local trainers of the CHEW mentors and VHWs in each local government area.

After assembling the start-up costs, we amortized them over a period of 3 years, applying a 3% discount rate. Three years was selected, because the CHWs were expected to serve a maximum of 3 years, at which time a new cohort would be trained. In addition, the manuals and job aids were expected to deteriorate or be lost within 3 years.

Data Analyses

Cost-Effectiveness Ratio Calculation

Change in Effectiveness: The goal was to contrast consultation levels pre- and post-CHW interventions. The impact assessment or post-intervention year was 2013 for all three models, but we only have pre-implementation outcomes with the 2012 consultation data for the VHWs and JCHEW-CBSD. For these two models we calculated the 2012 preimplementation unit outcomes per CHW by type. For the CV, the 2012 data is not a pre-implementation period, as one-third of the CVs were already working. Their 2012 and 2013 data were normalized to the actual number of CVs working in the state in each year. We measured only consultations that could have beenlinked to referrals from the CHW, namely ANC consultations, skilled birth attendance, and immunizations at birth, which is a proxy for newborn health examinations. First, we calculated the total number of consultations by type per clinic served by each cadre of CHWs in each state. In clinics where only the VHWs or CVs were affiliated, all consultations were attributed to them. For the clinics with JCHEW-CBSD, the service areas overlapped with CVs. The PRRINN-MNCH population-based survey of 2013 showed that where both were active, 28% reported interacting with the CV, while 53% reported interacting with the JCHEWCBSD. Therefore, in the clinics where both were active, the consultations were allocated based on these percentages. We then summed up the total number of calculations in each year by type of CHW providing support to that clinic. We then divided through by the number of CHWs (by cadre) actually supporting the assessed clinics.

We tested for significant differences in the patterns of monthly consultations from 2012 to 2013 with the Chi-square test. Because of the fractions reported per CHW, the monthly consultations by type of health outcome were multiplied by ten, so that the Chisquare test was for the difference in outcomes if there were 10 CHWs per clinic.

Cost-Effectiveness Ratios

We calculated the cost-effectiveness ratio two different ways. First, we calculated the unit health outcomes per $1000 invested in each type of CHW. The unit health outcomes were the average annual consultation rates per CHW for 2012-13 years for the CV, while for the VHW and JCHEW-CBSD they were per CHW for 2013 only. This is a variant of the cost-utility ratio, used for assessing how the utility of investments [30, 31].

Effectiveness Cost Ratio (ECR) = (Annual Clinical Outcomes per clinic per CHW) * 1000 / (Annual Unit Costs per CHW).

Following the recommendations of the World Bank to use the incremental cost-effectiveness ratio when determining the added health benefits when new services are introduced [31], we next calculated the incremental cost-effectiveness ratio for VHWs and JCHEW-CBSD, using 2012 as the pre-implementation year. The post-implementation outcomes were assessed as the change in outcome from 2012 to 2013. For the CVs, who were already operational in 2012, comparable 2010 data were not available, so we were not able to calculate the ICER for CVs. Because we are estimating the additional benefits gained when adding a CHW, the change in annual costs becomes equal to the annual costs observed after implementation of the CHW program.

Incremental Cost-Effectiveness Ratio (ICER) = (Annual Unit Costs per CHW)/(Clinical Outcomes 2013 – Clinical Outcomes 2012)

All calculations were based on the data presented in Tables 4 and 5.

Each state’s Operations Research Advisory Committee, the state-approved research ethics committees, approved the study protocol.

Results

Description of the CHWs

There were clear differences in the responsibilities, training, and support for each CHW model, as shown in Table 1. The CVs had the most minimal recruitment criteria, training, and associated costs. They received no stipends, while the JCHEW-CBSDs had the most extensive training, support and associated costs. VHWs generally fell between these two.

---

---

The most salient differences in how the three CHWs worked were the following:

• All VHWs were female, while CVs and JCHEW-CBSD were female and male. Female VHWs only worked with other women in the community. Among the CVs, each gender worked with community members of the same gender, while among the JCHEW-CBSD, males also provided support to female community members.
• CVs and VHWs were residents of the communities in which they worked, while the JCHEW-CBSDs were not. JCHEW-CBSDs came only weekly to each community.
• CVs and VHWs received about the same amount of training. JCHEW-CBSDs had the most extensive training, both before being recruited for the position and then specifically for CBSD activities.
• CVs worked primarily with groups, leading the series of discussions with the 15-20 individuals with whom they were working at the time. While they made home visits, these were only to follow-up with pregnant women. In contrast, the VHWs and JCHEW-CBSDs worked primarily through home visits. Some of the JCHEW-CBSD home visits were actually visits conducted at one central location in each community, generally a separate room within the village elder’s residential compound.
• CV and VHW responsibilities were primarily educational and motivational, while the JCHEW-CBSD provided education and direct service delivery for antenatal care, preventive care, and primary care for basic childhood illnesses. JCHEW-CBSDs carried a full kit to support delivery of care, including scale for weighing and essential drugs.
• CVs and VHWs used tally sheets for reporting their activities to their supervisors, and the connection between their activities and consultations based on their referrals was made by their supervisors, but not directly observed in the facility consultation registers. The JCHEW-CBSDs used logbooks, but they also entered their consultation data directly into health system registers, allowing direct tracking of consultations included as part of their activities.

Levels of Interaction with the Community by CHW Model

Each state averaged 1727 active CVs per year, 96 VHWs, and 50 JCHEW-CBSD. As shown in Table 2, in each state the CVs had contacted 70,000 villagers, and the VHWs and JCHEW- CBSDs each had met with around 20-25,000 individuals. The CVs reached 23,543 pregnant women with education about antenatal care, danger signs and birth preparations, while the VHWs and the JCHEW-CBSDs provided comparable education to 12,353 and 6,121 pregnant women, respectively. Of the three CHW models, the JCHEW-CBSDs had the highest rate of referrals for skilled birth attendance, both in absolute numbers (6121) and unit referrals (122).

One of the major contributors to the overall impact of the CVs in terms of interactions was their sheer numbers. After controlling for the number of CHWs in each model, the VHWs and JCHEWCBSDs have significantly higher levels of contact with individuals, numbering in the hundreds per year as compared to only 40 for each CV. Comparing the JCHEW-CBSD and the VHW, the JCHEW-CBSD also have about double the level of contacts and interactions, except for educating pregnant women about ANC and birth preparations, where they have almost identical levels of interactions.

---

---

Individual Health Outcomes by Interactions with a CHW

As shown in Table 3, compared to women who had not interacted with any CHWs, women who interacted with a CV, JCHEW-CBSD or VHW were significantly more likely to adopt recommended maternal and child health care practices. They were also more likely to know maternal, newborn and child danger signs. Relative to women who had not interacted with a CHW, there was a 2-3 times greater level of knowledge or behavior among women who had interacted with a JCHEW-CBSD or VHW. Comparing the JCHEW-CBSD with the VHW shows that interactions with the VHW were associated with the highest proportions of women adopting recommended maternal and child care practices, while interactions with the JCHEW- CBSD had more of an impact on newborn care practices than interactions with the VHW. All differences in the patterns by CHW were significant except for knowledge of 4 or more newborn danger signs, for which there was little difference associated with the type of CHW with whom the woman interacted.

---

---

Annual Facility Consultation Rates per CHW by CHW model

Table 4 presents the average annual number of consultations by health outcome per CHW for each type of CHW. Across all three CHW models, the average annual number of consultations attributed to a single CHW roughly doubled from 2012 to 2013, with the exception of one outcome, newborn examinations as measured by OPV at birth for the JCHEW-CBSD model. The lowest levels of consultations per CHW were at the clinics affiliated with the CVs, where the average annual consultations for any of the targeted activities were around ten times lower than at clinics associated with the VHWs or the JCHEW-CBSD. For the CVs, there was no significant difference between the 2012 and 2013 consultations by type (Chi-square=2.68, p=.614). From pre- to post-implementation, the VHWs were associated with the largest and significant gains in consultations at the clinics to which they were affiliated (Chi-square=27.19, p<.001). After implementation of the VHW program in 2013, the average annual number of ANC consultations at each clinic per VHW increased from 153.2 to 281.1 ANC consultations, and there was almost a tripling of deliveries at the clinic, from 12.2 to 33.3 per VHW per year. The JCHEW-CBSDs were associated with the highest average annual consultation rates, and the overall increase in consultations from 2012 to 2013 was significant (Chi-square=63.65, p<.001) However, the introduction of the JCHEW-CBSD in 2013 did not result in as much increase in any of the targeted consultations as for the VHWs, and polio vaccinations at birth at clinics supported by the JCHEW-CBSDs actually declined from 89.9 to 81.2.

---

---

Annual Unit Costs by CHW Model

As shown in Table 5, total annual costs for each of the CHW were similar for the CVs and JCHEW-CBSDs ($219,474 for the CVs and $222,159 for the JCHEW-CBSDs) and somewhat higher for the VHWs, $304,851. Unit costs differed more: $127 per year for the CVs , $3176 for the VHWs, and $4443 for the JCHEWCBSDs.

For virtually every cost category, the CVs had lower unit costs than any other CHW. The highest expense for the CVs was personnel. While the CVs did not receive any stipends, their supervisors did, and this accounts for the personnel cost for the CVs.

For the VHWs, the cost components contributing the greatest amount were recurrent costs ($1991 per VHW), comprised of reviews at state and national level, including planning, evaluation, and monitoring. The next highest cost for the VHWs was personnel, comprised of the stipends the VHW receive, as well as those for their CHW mentor/supervisors.

The highest unit costs for the JCHEW-CBSDs were for personnel ($3451), largely the JCHEW- CBSD salaries, but also their direct supervisors’ salaries.

---

---

Cost-Effectiveness Ratios by CHW Model

The relative superiority of the VHW model in terms of outcomes was also seen in the cost- effectiveness ratios shown in Table 6. The first panel of Table 6 gives the effectiveness to cost ratios for each model. All target health outcomes per $1000 invested in the CHW were greatest for the VHW. Each investment of $1000 is associated with almost 100 women coming to the clinic for a skilled birth delivery. The lowest returns per $1000 invested were found for the CV model, where each $1000 invested returns less than 10 consultations of any type. The unit health outcomes per JCHEW-CBSD were closer to those for the VHW, but the VHW still surpassed the JCHEW-CBSD in these outcomes.

The second panel of Table 6 reports the ICER for the VHW and JCHEW-CBSD models. The ICERs also show that the VHW program had the most favourable ICER for all of the measured health outcomes. The VHW ICER ranged from $75 per ANC visit to $152 per skilled birth attendant delivery, while for the JCHEW-CBSD the ICER were $34 and $449 for these two outcomes. Compared to the ICER for the JCHEW-CBSD model, the VHW ICER were one- third of the amount invested in the JCHEW-CBSD for the fourth ANC visit and for skilled birth attendant deliveries.

---

---

Discussion

There is much exploration going on in Nigeria and many other countries concerning which type of community health worker will have the most impact for any given level of investment. We were fortunate to be rolling out three variants of CHW models at roughly the same time period, 2012-13, in three Northern Nigeria states, and this provided an opportunity for us to compare the differences in cost-effectiveness across the three different models. While all three CHW models recruited CHWs to promote improved maternal, newborn, and child health care practices, they differed greatly in how they trained and supported the CHWs to achieve these changes. The series of analyses presented in this paper show that one of them, the VHW model, had better health outcomes for most measures and was the most cost-effective expenditure, compared to the other two models.

While all three CHW models focus on promoting MNCH services, CHWs participating in each model had widely varying levels of activity and intensities of community interactions. As shown in Table 2, the CVs interacted with the most individuals, but when normalized per CV, the JCHEW-CBSD had the highest rate of interactions with individuals, and they also made the most referrals for ANC visits and deliveries at the clinic by a skilled birth attendant. The analyses presented in this paper show, however, that these interactions did not necessarily translate into the highest rates of behavior change or visits to the primary health care center.

Women who had interacted with any of the three CHW models were significantly more likely to practice the recommended MNCH behaviors, compared with women who had not interacted with any of the CHW models. This is consistent with several of the systematic reviews of CHW programmes finding that women living in communities with CHWs were more likely to access primary health care services [23-25]. However, the data in Table 3 show that among women who interacted with each of the CHWs, interactions with the VHW were linked to greater increases in knowledge of danger signs, adoption of recommended maternal and child health care practices, and actual clinical visits. Compared to the other CHW models, women who interacted with the VHWs increased their antenatal care visits the most, with 92% of the women having made at least one ANC visit and 70% having made four or more ANC visits. This is corroborated in a recent study detailing the impact of the SURE-P VHWs [9]. The JCHEW-CBSD only surpassed the JCHEW-CBSD for three newborn care activities, while interactions with the CVs generally were linked with less knowledge or behavioral change than the other two models.

We believe that part of the difference in how women responded to the CHWs is associated with how the CHWs were trained to interact with their fellow villagers. These differences were linked to the conceptual models underpinning the design for each of the CHWs.

The CVs were the centerpiece of the PRRINN-MNCH community engagement strategy, which emphasized community empowerment. The CVs were trained in community empowerment, and were expected to engage a group of their neighbours in learning about maternal care practices through group discussions about these practices. The discussions were participatory and emphasized learning through songs and activities that were easy to learn and remember. The CVs organized several group discussions with them over a period of months, until they had covered all twelve of the recommended health practices. Throughout this series, the participants were getting to know each other better, which made it possible for them to help each other learn. Thus, the CV impact was through group learning situations, where part of the impact of the CV was the message, but another part was enabling participants to see others like themselves learn and then to receive social support for learning the jingle or particular health message under discussion at that meeting.

The VHWs were also volunteers recruited from the community where they worked, but the bulk of their activities were devoted to home visits and one-on-one interactions with the 30 families with whom they interacted on a regular basis. While some of their activities included group education sessions, most of their time was spent in one-on-one visits or support to pregnant women, escorting them to clinics or helping them to practice the activities recommended in the VHW pictorial guides. Thus, the VHWs had more hands-on time with women to actually help them adopt the recommendations. On re-visits to the home, the VHW checked in with the woman to see how she was doing. Particularly at the end of her pregnancy, the VHW was there to help her make her way to the clinic for delivery or respond to any problems. We believe this emphasis on actual behavior change was a key element underpinning the impact of the VHW program.

The JCHEW-CBSD had a different mission than either of the other two CHW cadres. As members of the local health clinic, the JCHEW-CBSDs were basically bringing services to the doorstep of the community, addressing access issues among the most hard-to-reach communities. While they also were trained in health promotion using the same materials as the CVs, their interactions with community members were quite different, as they focused on delivery of basic maternal and child health care services. They only visited the villages weekly, and did not necessarily see the same women on each visit. Their interactions took place in a central location in the village, not in the woman’s home, and they had much less time or opportunity to coach women on the adoption of recommended practices. They provided antenatal care services directly and used the IMNCI protocol to assist women in providing the most appropriate care for sick newborns, infants, or children. This is likely why the JCHEW- CBSD model had the greatest relative impact on newborn and child health care activities.

The emphasis of the JCHEW-CBSD on service delivery was also a factor contributing to the high consultation rates for antenatal care, deliveries by a skilled birth attendant and newborn vaccination. Not only did these include the consultations that the JCHEW-CBSDs made during their weekly visits, but they also reflected the trust they inspired in the communities they visited. The JCHEW-CBSDs were members of the primary health care staff, and women seeing the JCHEW-CBSD were able to see her in the community and at the clinic. Further, those who did not specifically see the JCHEW-CBSD during her visits to the community could have heard about her by word of mouth and then proceeded to see the JCHEW-CBSD or another health worker at the clinic. This pattern is consistent with the style of CHW adopted in Ethiopia with its clinic-based health extension workers who work partly in the community and partly in the clinic [28].

While the JCHEW-CBSDs had favourable clinical outcomes, they were also the most costly of the three CHW models. The JCHEW-CBSD model had a unit cost of $4443, over $1000 more per CHW than the $3176 per VHW. The VHWs had slightly higher average training costs, with another contributor to the cost differential being their monthly stipend of $62. Similarly, the higher cost of the JCHEW-CBSDs reflects not only their higher level of training but also their monthly stipends averaging $186. With no stipends, the unit cost of the CVs ($127) was much lower, and comprised largely of their annual re-training and supervision costs.

The cost-effectiveness ratios were most favorable for the VHW model, across all of the key MNCH indicators. According to the effectiveness-cost ratios presented in Table 6, for every $1000 invested, the VHW had the highest number of associated consultations at the primary care center. The ICER also were most favourable for the VHW model. Compared to the consultations in the year prior to the implementation of the VHW, the ICER were lower for the VHW than for the JCHEW-CBSD model.

Taken together, both sets of cost-effectiveness analyses suggest that the SURE-P MCH VHW model is the most cost-effective CHW model for promoting maternal, newborn, and child health outcomes in rural underserved communities in Nigeria, assuming that our analyses based on Northern Nigeria data hold for the rest of the country. While the JCHEW-CBSD model was most comparable to the HEW of the Ethiopian model, often taken as a model for other CHW programs [27-29], our analyses show that this model is not cost-effective for the Nigerian situation.

There are several limitations to any cost-effectiveness study. The first concerns the data used to calculate the cost-effectiveness ratios. While we used a triangulation strategy to obtain data on the outcomes associated with each CHW model, there were some weaknesses in the reporting system which may have led to an under-reporting of outcomes. The VHW Tally Sheets were in short supply in the initial months, so some of their visits may not have been fully documented in the first months. In communities where both the CVs and JCHEWCBSDs overlapped in service provision, the consultations were allocated to each model according to the proportions reporting interactions with the CV and JCHEW-CBSD in the Endline survey. This excludes 19% of the consultations which could have been with both CHWs. While this avoids double counting of consultations, the potentially reinforcing effect of CV plus JCHEW-CBSD is lost in the analysis of the CV and JCHEWCBSD outcomes. Some participants may have gone directly to the nearest hospital rather than their closest facility for the delivery of their infant. This was likely to be the case in the LGAs bordering on the state’s capital, as well as for any woman with danger signs for whom it was recommended to go directly to a hospital with comprehensive obstetrical care. Finally, polio vaccinations may have been delivered during the annual mass immunization campaign, and hence not reported at the clinic. Our choice of the newborn dose aims to minimize this reporting problem, so we expect that to be less of a problem than if we had used a higher dose of the polio vaccine. A second set of limitations pertain to the cost data. Due to the nature of the multi-state and national programming budgets, we discovered that the unit costs per CHW per model were fixed by design across all states for each model. This was a reasonable assumption for the VHW and JCHEWCBSD, which had fixed recruitment targets per state, but for the CV there was more variability in the numbers of CVs recruited and trained per year. We had to make assumptions based on the reports of the numbers of CVs actually working, but to the extent that the numbers actually recruited and working differed from the reports from each state, the unit costs could have been off for the CVs. In addition, we excluded from the cost calculations any training the CHW received prior to being recruited for the program under study. This excluded the not insubstantial costs of training to obtain the JCHEW certificate, and if these costs were included, they would have significantly increased the JCHEWCBSD unit cost. Another limitation relates to the method used to calculate the incremental cost-effectiveness ratios. Because we did not have comparable pre-implementation for the clinics served by the CVs, we could not calculate the ICER for the CVs. In addition, the ICER we calculated is based solely on the addition of the CHWs, so the pre-implementation cost is zero (no CHWs), while the post-implementation cost is the cost of one CHW for one year. This is different from the standard ICER formula which has total costs of the facility with and without the intervention in the numerator. These total costs were not available to us, and in any case were not the focus of this study. We were interested in the marginal change in consultations per Naira expended per CHW, therefore all our calculations of both costs and outcomes were normalized per CHW linked to the designated sites. Finally, only three of the northern states were involved in this study, and the findings may not be applicable for all of Nigeria. While we believe that the findings pertaining to the VHW model are applicable at a national level, as this has been a national program, the other two programs were multi-state, and replication of these models in other states could entail a different cost profile.

Despite these limitations, we believe that our cost-effectiveness study of CHW models in northern Nigeria provides an insight into the type of CHW model that is most appropriate for the smaller rural communities where they are likely to work. The SURE-P VHW model was shown to be the most cost-effective model with higher promise on return on investment if scaled up in Northern Nigeria where the need is greatest for health care workers, particularly in rural communities.

This is particularly welcome news for Nigeria, as the SURE-P VHW model also has demonstrated its scalability. While this study only reports on data from the VHWs working in three states, it has been a nationwide program, operating with about 6000 VHWs affiliated with 1000 PHCs in every Nigerian state and the Federal Capital Territory. As in the three northern states reported here, all the PHCs selected to participate were in rural areas with underserved maternal and child health needs. National evaluation studies demonstrate that the program’s components were implemented successfully across the nation. While not explicitly focusing on the contribution of the VHWs, as in this study, they also found increases in antenatal care visits, deliveries with a skilled birth attendant, and newborn vaccinations [33, 34]. The SURE-P MCH program was suspended at the end of 2015 after the oil prices fell. However, the federal government restarted the Midwife Service Scheme in 2016, and this would be an excellent time to also re-launch the VHW program as part of the national CHW program.

As part of the National Primary Care Development Agency, the VHW model developed by the SURE-P MCH program has several features which will make it sustainable, compared to the other models. First, the VHWs are linked directly to the health system, which already employs and trains the midwives and CHEWs who provide supervision and receive the referrals from the VHWs. There is no need for a parallel supervisory or employment structure. Second, the incentives and support to the midwives are incorporated into the nationwide program, the Midwife Service Scheme, which supports the training and posting of midwives to the underserved PHCs targeted by the VHWs under the SURE-P MCH model. Third, the program had no difficulty in locating eligible candidates for the VHW work, and feedback from the VHWs showed that they were uniformly enthusiastic about their work, wanted to continue it, and believed it should become the national CHW program [9]. Fourth, the funding for the SURE-P MCH program came completely from Nigeria’s oil revenues. Although the country is currently undergoing a review of its health funding priorities, even at their reduced levels the oil revenues will continue to exist and can be used to support VHWs, consistent with the priority Nigeria has placed on reducing maternal and child mortality in the underserved communities of Nigeria. Indeed, the current crisis in Nigeria’s economy related to the drop in oil prices serves as a wake-up call to be more targeted and efficient in allocating the returns on the nation’s oil wealth. This cost-effectiveness study shows that investing in SURE-P MCH’s VHWs was highly costeffective at improving key maternal and child health outcomes contributing to Nigeria’s achievement of sustainable development goals of reduced maternal and child mortality.

Many countries have been exploring the most cost-effective way to expand their CHW workforce. If the relative differences in the three models examined here are valid in other countries, this study suggests that a scaled down adaptation, such as the SURE-P MCH VHW model, would be more cost-effective than the more comprehensive CHW model recommended in the One Million CHW report, which is closer to the JCHEW-CBSD model examined here.

Acknowledgements

We thank the Women for Health programme, funded and supported by the UKAid from the UK Government for its support. Health Partners International (HPI), Save the Children International and GRID Consulting manage the programme. The authors thank the SURE P MCH programme for its kind support to the study including assistance with consultation and cost data. We also thank the programme officers, many Village Health Workers, Community Volunteers, JCHEW-CBSDs and others who gave their time freely to participate in this study.

References

1. Naicker S, Tutt RC, Eastwood JB. Shortage of healthcare workers in developing countries--Africa. 2009;19(1 Suppl):S1-60.
2. United Nations. Sustainable Development Goals: The Road to Dignity, UN Secretary General’s Office, 2015, New York, NY.
3. Nigeria Demographic Health, Survey 2013. National Population Commission, Federal Republic of Nigeria. ICF-Macro: Rockville, USA; 2014.
4. Fapohunda BM, Orobatan NG. When women deliver with no one present in Nigeria: Who,What, where and so what? PLoS One. 2013;8(7):e69569.
5. Uneke C, Ogbonna A, Ezeoha A, Oyibo P, Onwe F, Ngwu B. The Nigeria health sector and human resource challenges. Internet J Health. 2007;8(1).
6. Ezeonwu MC. Nursing education and workforce development: Implications for maternal health in Anambra State, Nigeria. Int J Nurs Midwifery. 2013 Apr;5(3):35-45.
7. Omoleke II. The primary health services in Nigeria: Constraints to optimal performance. Niger J Med. 2005;14(2):206-212.
8. Nkwo PO, Lawan LO, Ubesie AC, Onodugo VA, Obu HA, Chinawa JM. Poor Availability of Skilled Birth Attendants in Nigeria: A Case Study of Enugu State Primary Health Care System. Ann Med Health Sci Res. 2015 Feb;5(1):20-25.
9. Findley SE, Afenyadu G, Okoli U, Baba H, Bature R. Implications of the SURE-P MCH National Village Health Worker Experience in Northern Nigeria for the Road Map for Village Health Workers in Nigeria. J Community Med Health. 2016;6:419. doi:10.4172/2161-0711.1000419.
10. Mangham-Jefferies L, Pitt C, Cousens S, Mills A, Schellenberg J. Cost-effectiveness of strategies to improve the utilization and provision of maternal and newborn health care in low-income and lower-middle-income countries: a systematic review. BMC Pregnancy Childbirth. 2014 Jul 22;14:243.
11. McPake B, Ijeoma Edoka, Sophie Witter, Karina Kielmann, Miriam Taegtmeyer, Marjolein Dieleman, et al. Cost-effectiveness of community-based practitioner programmes in Ethiopia, Indonesia and Kenya. Bull World Health Organ. 2015 Sep 1;93(9):631-639A.
12. Singh, Prabhot. One Million Community Health Workers: Technical Task Force Report, 2011. Earth Institute, Columbia University.
13. Bhutta Z, Lassi ZS, Pariyo G, Huicho L. Global experience of community health workers for delivery of health-related millennium development goals: A systematic review, country case studies, and recommendations for integration into national health systems. World Health Organization, Global Health Workforce Alliance, Geneva, Switzerland; 2010.
14. United Nations, Secretary General’s Office. Strengthening Primary Health Care Centers through Community Health Workers: Investment Case and Financing Recommendations Report 2015.
15. Lehmann U, Sanders D. Community health workers: What do we know about them? The state of the evidence on programmes, activities, costs and impact on health outcomes of using community health workers. World Health Organisation; 2007.
16. Walker DG, Jan S. How do we determine whether community health workers are cost effective? Some core methodological issues. J Community Health. 2005 Jun;30(2):221-9.
17. Phillips JF, Bawah AA, Binka FN. Accelerating reproductive and child health programme impact with community-based services: the Navrongo experiment in Ghana. Bull World Health Organ. 2006;84(12):949-955.
18. Viswanathan M, Kraschnewski J, Nishikawa B, Laura C Morgan, Patricia Thieda, Amanda Honeycutt, et al. Outcomes of community health worker interventions. Evid Rep Technol Assess (Full Rep). 2009 Jun;(181):1-144.
19. Bigirwa P. Effectiveness of community health workers (CHWs) in the provision of basic preventive and curative maternal, newborn and child health (MNCH) interventions: A Systematic Review. Health Policy Develop. 2009;7:162-172.
20. Jafar TH, Islam M, Bux R, et al. Cost-effectiveness of community-based strategies for blood pressure control in a low-income developing country: findings from a cluster-randomized, factorial-controlled trial. Circulation, 2011; 124 (15):1615-1625.
21. Viswanathan K, Hansen PM, Rahman MH, Steinhardt L, Edward A, Arwal SH, et al. Can community health workers increase coverage of reproductive health services? J epidemiol community health. 2012 Oct;66(10):894- 900.
22. Ramsey K, Hingora A, Kante M, Jackson E, Exavery A, Pemba S, et al. The Tanzania Connect Project: A cluster-randomized trial of the child survival impact of adding paid community health workers to an existing facilityfocused health system. BMC Health Serv Res. 2013;3(Supp l )2:S6.
23. Vaughan, Kelsey, Kok, Maryse C, Witter, Sophie and Dieleman, Marjolein. Costs and costeffectiveness of community health workers: Evidence from a literature review. Hum Resour Health. 2015 Sep 1;13:71. Doi 10.1186/s12960-015-0070-y
24. Tani K, Exavery A, Baynes CD, Pemba S, Hingora A, Manzi F, et al. Unit cost analysis of training and deploying paid community health workers in three rural districts of Tanzania. BMC Health Serv Res. 2016 July 8;16(1):1-7.
25. Pence BW, Nyarko P, Phillips JF, Debpuur C. The effect of community nurses and health volunteers on child mortality: The Navrongo Community Health and Family Planning Project. Scandinavian J Public Health. 2007;35(6):599-608.
26. Torrance GW, Drummond MF. Methods for the economic evaluation of healthcare programs. 3rd ed. Oxford University Press; 2005.
27. Banteyerga H. Ethiopia's health extension program: Improving health through community involvement. MEDICC Rev. 2011 Jul;13(3):46-49.
28. Dynes M, Buffington ST, Carpenter M, Handley A, Kelley M, Tadesse L, et al. Strengthening maternal and newborn health in rural Ethiopia: Early results from frontline health worker community maternal and newborn health training. Midwifery. March 2013;29(3):251-259.
29. Medhanyie A, Spigt M, Kifle Y, Schaay N, Sanders D, Blanco R , et al. The role of health extension workers in improving utilization of maternal health services in rural areas in Ethiopia: a cross sectional study. BMC Health Services Res. 2012 Oct;12(1):352.
30. Cohen DJ, Reynolds MR. Interpreting the Results of Cost-Effectiveness Studies. J Am Coll Cardiol. 2008;52(25):2119-2126. doi:10.1016/j. jacc.2008.09.018
31. Jamison DT . Cost-Effectiveness Analysis. In: Jamison DT, Breman JG, Measham AR, editors. Priorities in Health. International Bank for Reconstruction and Development/The World Bank: Washington, DC; 2006.
32. Tan-Torres Edejer, Baltussen R, Adam T, Hutubessy R, Acharya A, Evans DB, et al. Making Choices in Health: WHO Guide to Cost-Effectiveness Analysis. Geneva: World Health Organization; 2003.
33. World Bank Group. The impact of the Subsidy Reinvestment and Empowerment Programme on Maternal and Child Health on Skilled Birth Attendance and Antenatal Care. Washington, DC: Development Impact Evaluation Group, World Bank Research Group; 2015.
34. Nsofor, Ifeanyi M, Adaobi N, Ezeokoli, Kachikwulu O Adabike, Ike Anya, Chikwe Ihekweazu. An evaluation of the maternal and child health project of the Subsidy Reinvestment and Empowerment Programme (SURE-PMCH). Abuja: EpiAFRIC. 2015 May;