|1. Project Data:
ICR Review Date Posted:
|India: Reproductive & Child Health Second Phase
Project Costs(US $M)
Loan/Credit (US $M)
|Health, Nutrition and Population
Cofinancing (US $M)
|DFID, Foreign sources (unidentified), UN Fund for Population Activities
Board Approval Date
|Health (97%), Other social services (1%), Sub-national government administration (1%), Central government administration (1%)|
|Population and reproductive health (29% - P)
Child health (29% - P)
Health system performance (14% - S)
Other social development (14% - S)
Decentralization (14% - S)|
||ICR Review Coordinator:
|Denise A. Vaillancourt
||Judyth L. Twigg
||Christopher D. Gerrard
|2. Project Objectives and Components:|
Schedule 2 of the Development Credit Agreement states that
The objective of the Project is to support the Borrower’s continuing Program for Reproductive and Child Health (the RCH II Program) that aims to achieve the reduction in maternal mortality and child mortality.
The Project Appraisal Document/PAD (p. 5) states that
The Project Development Objective (PDO) is to expand the use of essential reproductive and child health services of adequate quality with reduction of geographical disparities.
Proxy indicators focused on the following essential services: contraceptive services; deliveries conducted by skilled personnel; child immunization; post-natal visits; and polio eradication. The PAD (p. 5) also notes that
This proposed operation would help reduce maternal and child mortality and morbidity, lower fertility and the rate of population growth.
This ICR Review will assess project performance against the higher-level objectives of the DCA (also mentioned in the PAD), as well as the service-level objectives and accompanying indicators specified in the PAD that contribute to those higher-level objectives.
b. Were the project objectives/key associated outcome targets revised during implementation?
The RCH Project II was designed to support Government of India’s (GoI) National Reproductive and Child Health Program 2005/06-2009/10 (the RCH Program-Second Phase). The project’s three components (including their estimated and actual costs) are presented below.
1. Improvement in Essential RCH Services (Appraisal estimate: US$1,367 million; actual cost: US$3,081 million)
1.1 Activities Administered by the Ministry of Health and Family Welfare (MoHFW). This subcomponent was to support: (a) procurement of goods and services, particularly drugs, medical supplies and equipment; (b) routine immunization (including vaccines, cold chain equipment and maintenance, among other supports); (c) behavior change communication; (iv) training for improving technical and implementation skills; (v) expanding public private partnerships, with advice and support to states to this end; and (vi) policy development and pilots (private provider accreditation; social franchising, innovative financing schemes).
1.2 Innovative and Developmental Activities in State Program Implementation Plans (SPIPs). This subcomponent was to support innovative activities in states’ plans and provide flexible funds to support states’ creativity. Not defined ex-ante, this component aimed to support initiatives to expand access to family planning, safe motherhood services, newborn care, community-based child health and nutrition programs, promotion of adolescent health and development, urban RCH care, and service provision to special populations (tribal and scheduled caste).
2. Technical Assistance, Monitoring and Evaluation (Appraisal estimate: US$26.7 million; actual cost: US$30.0 million)
2.1 Technical Assistance (TA). Against the backdrop of a newly established MoHFW National Health Systems Resource Center (NSHRC) for the purposes of harmonizing TA contracting, this component aimed to support (a) states’ (especially Empowered Action Group/EAG and North-Eastern) planning capacity and (b) implementation of GoI’s comprehensive Governance and Accountability Action Plan (GAAP) to strengthen procurement capacity (both GoI and agent/UN agency). This component received parallel financing from DFID under a plan jointly agreed by GoI, DFID and IDA.
2.2 Monitoring and Evaluation. This subcomponent was designed to support a comprehensive M&E system to enable the close and regular monitoring of progress against Results Matrix indicators. Support would include two rounds of the Reproductive and Child Health Rapid Household Survey (2006-07 and 2009-10) to complement service statistics and community consultations. Special studies and polio eradication monitoring were also envisaged.
3. Polio Eradication (Appraisal estimate: US$829 million; actual: US$1,364 million)
This component aimed to support India’s efforts to achieve polio-free status, filling the program’s financing gap. A precursor to polio eradication, polio-free status was expected to be achieved by 2007, with subsequent supplemental surveillance activities envisaged for support for a few years after in order to declare polio eradication.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Costs. The actual project cost of US$4,475 million was double the appraisal estimate. Component 1 (essential RCH services) cost over twice the original estimate; Component 2 (TA and M&E) was 12 percent more expensive than planned; and the cost of Component 3 (polio eradication) exceeded the original estimate by two-thirds. The ICR does not explain the significant differences between planned and actual costs for Components 1 and 3, but they are likely to be associated with the two-year extension of the project (and program) (i.e., two additional years of operating costs).
Financing. The Bank, DfID and UNFPA pooled their funding and jointly disbursed against a broad subset of RCH II program expenditures, in accordance with pre-determined percentages through the RCH Project II. The IDA credit of 245 million SDRs was fully disbursed. DFID disbursed about 60 percent and UNFPA 125% of their respective planned financings (as expressed in US$ in Annex 1 of the ICR). Against the appraisal estimate of US$338 million in “unidentified foreign sources,” only US$25 million was disbursed (Annex 1 of the ICR), but the financier is not identified.
Borrower Contribution. GoI’s actual contribution of US$3,820 million was three and a half times the appraisal estimate of US$1,062.
Dates. The project was restructured in May 2010, including: a revision of the Results Framework and indicators (but not a change in PDOs); an extension of the project’s closing date by 18 months from September 30, 2010 to March 31, 2012; and a reallocation of the proceeds of the Credit to Category 2 for polio operating costs. Amendments to the DCA in 2008 (March and October) redefined categories to allow the use of Bank financing for operating expenses, TA for implementation of the Governance and Accountability Action Plan, and contractual staff and consultants. A fourth amendment to the DCA (March 2012) reallocated proceeds.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:
High.The PDOs are highly relevant to India’s 12th Five-Year Plan (FY2013-17), whose overarching objective is faster, sustainable and more inclusive growth. The Plan advocates greater attention to inequality – both in outcomes and in opportunities – and ensuring access to good-quality health services. It sets specific targets for decreasing the rates of infant and maternal mortality (Country Partnership Strategy 2013). The PDOs are also highly relevant to the March 2013 World Bank Group’s Country Partnership Strategy, which is organized under three engagement areas: integration, transformation, and inclusion. Under “inclusion,” the Bank commits to strengthened public and private health delivery systems through the channeling of a larger share of health interventions to the state level, with an emphasis on disadvantaged states. The CPS focus is on strengthening institutions and accountability, developing local systems and capacities, private-public partnerships, and improved access and affordability for the poor and vulnerable. The PDOs (along with India’s Plan and the Bank’s Strategy) are well aligned with the country’s health issues: inequity of health status, with maternal and child mortality highest among the poorest and most vulnerable, persistent issues of service quality and inequitable access, and uneven capacity and performance across states.
b. Relevance of Design:
Substantial. The project’s operational objectives – to expand and improve key reproductive and child health services and make them more accessible and used, especially by poorest and most vulnerable groups, were directly supportive of the high-order objectives of reducing maternal and child mortality. The project’s support to key reproductive and child health services was appropriate to this end: contraceptive services, pre-natal care, deliveries by skilled personnel, post-natal care, and child immunization including efforts to eradicate polio. The project’s support to improving the availability, quality and utilization of these services – in terms of inputs and outputs – were also appropriate. They focused on both the supply-side (procurement of goods and services for the programs and services, such as vaccines, drugs, training, equipment, renovations and upgrading) and the demand-side (behavior change communications and financial incentives and other innovations to incite use). In addition, the design provided flexible funds to the states, with an emphasis on the most disadvantaged, to enhance their decision-making and resource allocation autonomy, in complement to state-level capacity building interventions. Project activities were appropriately oriented toward reducing geographic disparities by weighting funding in favor of the Empowered Action Group States, plus five additional states with poor maternal and child health status, and the north eastern states, which were generally worse off than other states as measured by socio-economic indicators. The design also appropriately embraced a program-wide approach, seeking to coordinate and consolidate external support around nationally articulated objectives and programs.
|4. Achievement of Objectives (Efficacy) :|
The program (and project) uses three main data sources: DLHS2 2002-04 for the baseline, DLHS3 2007-08 for the mid-point, and a UNICEF Coverage and Evaluation Survey (CES) in 2009 for an additional year of update. A DLHS4 was not conducted in time to provide end-of-project data.
Trends documented with these three sources are problematic for evaluating this project. First, they do not overlap sufficiently with the program implementation period. The program was launched in April 2005 and was extended to 2012, covering a seven-year period. DLHS 2 and DLHS3 document trends over the first half of the program period. The failure to complete the DLHS4 has resulted in the absence of trend data to measure progress during the second half of the program period.
Second, the comparability of the DLHS3 and the CES 2009 for assessing trends is uncertain. The Borrower, in its section of the ICR (p. 52), notes that the two sources are “not strictly comparable,” and this has been corroborated by the TTL, who notes that the sample sizes are completely different. Nevertheless, the ICR is pragmatic in drawing on this extra year of data and attempting to triangulate derived trends with its field visit findings.
The ICR is appropriately focused on program-level evaluation and the project’s contribution thereto, rather than on the project and attribution. Even though project implementation and disbursements were seriously delayed by one year or more after the project was declared effective in November 2006 due to a range of issues (ICR p. 8), the Bank can be credited with important contributions to the program from its very outset in 2005 on two fronts: (1) its substantial non-financial support during design and start-up of this program-wide approach, especially: technical and strategic dialogue with the Government; support to the consolidation of RCH II and the flagship National Rural Health Mission; the mobilization and convening of the support of other partners, both pooling and others; the active participation in Joint Review Missions; and extensive technical advice and support on governance issues; and (2) the project’s retroactive financing provision that allowed the reimbursement of operating expenses covering the early years of project implementation.
PDO: Expanded use of Essential Reproductive and Child Health Services, with reduction of geographic (and other) disparities: Modest achievement
(Despite apparently substantial achievement registered over the first half of the project, the absence of data/trends for the remainder of the project has caused a downgrade to modest.)
In the absence of end-program data, the ICR (p. 17) notes that after the mid-term review, project implementation geared up and made important contributions to the above-cited program goals, which would lead them to conclude that targets are likely to have been met or exceeded. Among the major contributions are:
In support of service expansion:
In support of reaching vulnerable groups and disadvantaged states:
- 25 states/union territories contracted the non-government sector to improve delivery of essential RCH services, surpassing the target of 15 states.
- In 50 percent of districts in EAG states and Assam, the district hospitals conducted at least 20 C-Sections per quarter, falling short of the project target of 100 percent of districts in those states.
- In 40 percent of districts at least one sub-district hospital conducted 10 C-Sections per quarter, falling short of the project target of 100 percent of districts.
- 38 percent of 24-hour primary health care centers conducted more than 10 deliveries per month, not meeting the target of 60 percent.
- Access to emergency obstetric care was enhanced through the establishment of first referral units (FRUs) and 24-hour services at primary health centers (PHCs). 8,475 PHCs were made operational 24x7 by March 2012, as against 1,263 in March 2005; and 2,315 FRUs were functional in March 2012, as against 955 in March 2005. The facilities providing these services were upgraded, but there remain questions about their quality (ICR p. 9).
- 32,291 nursing and midwifery personnel were trained in skilled birth attendance as of September 2011. 1,070 medical officers were trained in life-saving anesthesia skills, and 601 medical officers were trained in comprehensive emergency obstetric care, including C-Sections. Ten-day trainings on basic emergency obstetric care have been initiated in the states. Master trainers are available in most states.
In support of demand stimulation:
- 100 percent of state plans contained specific activities to reach vulnerable groups, surpassing the target of 75 percent (2009).
- The number of rural health facilities and providers was increased. Additional auxiliary nurse midwives and nurses were trained, more than 600,000 community health workers (ASHAs) were recruited, and facilities were expanded to offer emergency obstetric care and blood bank/storage units.
- Improved access to care for vulnerable groups was supported through the work of community health workers’ outreach and a cash transfer scheme to encourage women to deliver in health facilities.
- Mobile health clinics in trial blocks were found to have increased access to RCH services. However, the ICR also notes (p. 25) a lack of effective trainers for enhancing sensitivity to gender and equity issues and significant variation across states in prioritization of services for vulnerable groups.
Two synergistic interventions were found to be effective in imparting the positive trend in the use of assisted deliveries/institutional deliveries.
Because of low state capacity, IEC activities planned under the program were only partially implemented (mostly mass media campaigns, with behavior change communications relatively neglected). IEC was thus assessed not to be an important contributing factor to improved use of services.
- Under RCH II a conditional cash transfer scheme provided poor, vulnerable pregnant women with $35 to cover the expenses associated with their care during and following their pregnancies, including the delivery.
- Under the NRHM a newly established cadre of community health workers, comprised of young women from the villages, counseled families, encouraged the use of services, and improved access to the above-cited conditional cash transfers. These workers were rewarded with $4-5 under the conditional cash transfer for every referral.
In support of quality improvements:
In support of state-level capacity building:
- Quality Assurance Committees were constituted and notified at the state and district levels in all states. Many states have developed checklists for monitoring. Several individual states established initiatives to improve the quality of RCH services: Gujarat has a quality assurance manual for RCH with a focus on total quality management and accreditation of PHCs and community health centers; and Madhya Pradesh and Odissa set up quality assurance programs, wherein district hospitals were strengthened to meet national accreditation standards. The National Health Systems Research Center successfully completed certification of 80 health facilities across the various states.
- 55 percent of districts conducted training in the last three months of the project for skilled birth attendants, and 35 percent on integrated management of neonatal and childhood illness. This indicator was added during restructuring and neither baselines nor targets were specified.
- The ICR reports that by closing all districts had at least one month’s supply of critical inputs (measles vaccine, oral contraceptive pills and gloves). Field visits indicate that essential drugs are now available in the system, but it is not clear if this indicator is regularly tracked.
- Community monitoring, operational in 9 states by the time of the project’s closing, is noted by the Borrower to have revealed improvement in the quality of service delivery, infrastructure and service utilization, though no data are provided.
- However, both DfID and UNFPA highlight quality improvement as an unfinished agenda (ICR, pp. 67-68).
In support of central-level capacity building:
- 35 states/Union Territories successfully completed the institutional mobilization phase by 2010, meeting the target.
- All EAG and NE states were visited by the MoHFW state facilitation teams, meeting the target.
- Significant fiduciary capacity was built.
In support of polio eradication efforts:
- Both the program-wide approach and the Detailed Implementation Review (DIR: a review of allegations of corruption in procurement under RCH I; see Sections 5 and 11.b) have culminated in improved capacity for program management, more proactive monitoring and support of high-priority states, and improved capacity in procurement and financial management.
- Stool samples were collected from at least 90 percent of acute flaccid paralysis cases within 14 days, surpassing the target of 80 percent.
- The target of non-polio acute flaccid paralysis rate of at least one case per 100,000 children below 15 years of age (an indicator of surveillance sensitivity) was reached, with a rate of 6.32 per 100,000 children.
- Early project (2002-04 to 2007-08) trends show improving trends in the use of any contraceptive method, during the first half of the project, increasing from 45 percent to 47 percent, against an end-of-project target of 52 percent.
- Modest increases for permanent methods (from 34 percent to 35 percent against a target of 36 percent) and for spacing methods (from 11 percent to 12 percent against a target of 16 percent) are reported in the ICR p. 18). It is not clear whether these very small changes of one percent are statistically significant.
- Rates for scheduled caste groups increased from 43 percent to 49 percent, exceeding the 45 percent target, while rates for scheduled tribal groups increased from 39 percent to 42 percent, approaching the target of 45 percent.
- EAG States were slated to increase from a baseline of 33 percent to 40 percent. By the mid-term four of the eight states had achieved the target, but the ICR does not present the baseline for each of the eight states.
- Expanded use of delivery services, with reduction of disparities, exceeded targets by the project’s midpoint. Available data show that trends were improving between 2002-04 and 2007-08, and continued to improve, as documented by the CES (2009) and corroborated by field visits in two states (MP and Karnataka) and by reviews of Joint Review Mission reports.
- Expanded use of delivery services conducted by skilled providers exceeded targets. Overall increases were from a level of 48 percent in 2002-04 to a level of 53 percent in 2007-08, rising to 76 percent in 2009, against the target of 60 percent.
- Reduction of disparities in the use of skilled delivery services also exceeded targets. The 2002-04 level for EAG states was 32 percent. Levels in 2007-08 had risen to a range of 25 to 53 percent, and these further increased to a range of 47 to 79 percent in 2009, indicating more important increases than for the general population. The overall level of 66 percent for EAG states achieved in 2009 exceeded the program target of 45 percent. Gains were also impressive for the lowest wealth quintile, whose use of skilled delivery services rose from 28 percent in 2002-04 to 48 percent in 2009 (no specific target had been set for this group and no 2007-08 data were provided).
- From an overall baseline of 35 percent, actual use of skilled delivery services rose in 2009 to 76 percent for Scheduled Castes and 61 percent for Scheduled Tribes, exceeding the target of 40 percent.
- Proportion of institutional deliveries also increased overall (from 40% in 2002-04 to 47 percent in 2007-08 to 73 percent in 2009), with most EAG states having increases greater than those for the overall population thus revealing important headway in reducing disparities.
- Expanded use of pre- and post-natal services, with reduction in disparities, was substantially achieved.
- The proportion of women with at least three antenatal visits increased overall (from 50% in 2002-04 to 50% in 2007-08, rising to 63 percent in 2009), with most EAG states having increases greater than the increase for the overall population, thus revealing important headway in reducing disparities.
- The percent of mothers and newborns visited within two weeks of delivery by a trained community-level health provider is reported to have increased (from less than 10 percent in 2002-04 to 50 percent in 2007-08, exceeding the target of 40 percent).
- The percent of children 12-23 months of age who were fully immunized was on the rise in the early years of program implementation.
- Between 2002-04 and 2007-08 DLHS data show that:
- Immunization of females increased from 44 percent to 60 percent (project target: 75 percent), and
- Immunization of males increased from 45 percent to 62 percent (project target: 75 percent).
- CES data reveal progress in equity of results between 2002-04 and 2009:
- Immunization of schedule caste groups rose from 42 to 59 percent (project target: 75 percent).
- Immunization of schedule tribal groups rose from 37 percent to 50 percent (project target: 75 percent).
- EAG states actually reached the project target of 60 percent in 2009, rising from 28 percent to 61 percent. Only two of the eight EAG states achieved the target by 2007-08 (DLHS3), and by 2009 all EAG states had achieved the target.
- Immunization rates among the lowest wealth quintile also increased from 31 percent to 47 percent (no project target set).
- Polio targets were exceeded under the program (validated data from National Polio Surveillance Project).
- By 2012 at least 98 percent of households with eligible children were covered during national and subnational polio immunization days in high-risk districts. This surpassed the target of 80 percent.
- India has been certified by WHO as having achieved polio-free status as of January 2012.
Maternal, infant and child mortality and total fertility are affected by a broad range of socio-economic factors, improved reproductive and child services being only one among many others. Given the timeframe of program implementation and the unavailability of health services and impact data after 2010, it is difficult to assess with confidence the extent of the program’s contribution to documented trends in these outcomes, as well as the trends in these outcomes themselves for the project's latter years. Nevertheless, because of the good design of the program, it is plausible to assume that the program might be contributing to these outcomes, especially in the later years when implementation was more accelerated, and in the post-project years. This can only be adequately assessed once trends in services and outcomes are established for the later years of the project (and post-project). Available trends are as follows (National Sample Registration Surveys – SRS):
- Maternal mortality declined from 301 in 2001-03 to 254 in 2004-06, further declining to 212 in 2007-09.
- Total fertility declined from 2.6 to 2.5 between 2008-10, with most states showing a decline of 0.1 and 0.2 points.
- There was an overall decline in infant mortality from 58 in 2005 to 47 in 2010, with wide variance across states.
- Under-five mortality has changed very little between 2008 (60) and 2010 (59), although some disadvantaged states with higher than national averages have shown significant rates of decline.
The project design encouraged allocative efficiency – of both Government budget and external pooled financing – on a number of fronts. First, it supported and prioritized high impact, cost-effective reproductive health services that were major components of national programs. Second, it emphasized the channeling of support to low performing, disadvantaged states with the highest rates of maternal, infant and child mortality, where the potential for impact was greatest for boosting India’s average rates. Third, its program-wide approach encouraged a coordination and consolidation of donor support through pooling of significant funding and coordination across a range of major donors. Fourth, the flexible funding to the states allowed them the opportunity to allocate resources to respond to the local challenges and context. Fifth, it enabled the mobilization of adequate support for polio eradication efforts and the allocation of a considerable portion of these funds to the two states where polio cases were still being reported (Bihar and Uttar Pradesh).
The program-wide design also encouraged the coordination/consolidation of big-picture activities and events, such as annual program reviews and joint missions at the central level, quarterly reviews and joint support to the states, and the coordination of massive amounts of technical assistance available for the program.
Inefficiencies in program implementation were encountered. There was a confusion of roles, responsibilities and procedures between the Government’s long-established RCH II program and its newly established, broader umbrella program National Rural Health Mission (NRHM), within which RCH II was an important component. Project support (both technical and financial) helped build and nurture synergies between the two programs, including the mobilization of significant additional funding, major policy and programmatic shifts, and enhanced supply- and demand-side interventions (ICR, p. 24). State and district capacity were initially overwhelmed with the management requirements of these two programs, which are gradually being consolidated.
Additionally, there were important setbacks in project implementation caused by the Detailed Implementation Review (DIR), which raised issues of potential corruption and required the negotiation of a detailed action plan to improve the rigors of fiduciary requirements and practice. There were weaknesses in procurement and financial management at both the central and state levels, including staff capacity and turnover. These short- to medium-term inefficiencies were a consequence of deliberate project design and post-DIR strategy, which chose to invest in the nurturing and use of national capacity systems and capacity; and they were anticipated in the risk assessment. The ICR (p. 7) provides evidence that through these DIR-related struggles and setbacks, system-wide capacity has been and continues to be built through a learning-by-doing process, and has culminated in improved use of project resources. Procurement standards are put into place to ensure the best value for money in the acquisition of goods and services, as well as transparency in the whole procurement process. The DIR uncovered irregularities in procurement under RCH I, which were undermining the efficient and transparent use of resources and raised concerns of potential corruption. They were addressed through capacity building at central, state and district levels, a decision to stop financing state-level procurements until state capacity was raised to a minimum standard and states would no longer resist post-reviews, and other health sector measures agreed with GoI to remedy and/or mitigate risks to fraud and corruption. While the DIR did slow down project implementation, the counterfactual of no DIR would have made the use of project resources much less efficient.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated