Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Bangladesh - Health Nutrition And Population Sector Program


  
1. Project Data:   
ICR Review Date Posted:
10/03/2013   
Country:
Bangladesh
PROJ ID:
P074841
Appraisal
Actual
Project Name:
Bangladesh - Health Nutrition And Population Sector Program
Project Costs(US $M)
 4306.00  4300.00
L/C Number:
C4052
Loan/Credit (US $M)
 300.00  293.40
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 387.90  387.80
Cofinanciers:
CIDA, EC, KfW, Sida, DfID, UNFPA
Board Approval Date
  04/28/2005
 
 
Closing Date
12/31/2010 12/31/2011
Sector(s):
Health (40%), Central government administration (36%), Other social services (11%), General public administration sector (9%), General education sector (4%)
Theme(s):
Nutrition and food security (25% - P) Health system performance (25% - P) Population and reproductive health (24% - P) Tuberculosis (13% - S) Child health (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judyth L. Twigg
Alain A. Barbu Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:
As stated in the Development Credit Agreement (DCA, p. 19), the project’s objectives were to: “(i) reduce infant, under-five, and maternal mortality and the proportion of malnourished children; (ii) eliminate the gender disparity in child malnutrition and mortality; (iii) ensure increased access to reproductive health services; (iv) lower total fertility with a view towards achieving replacement level fertility by 2010; (v) reduce the burden of tuberculosis, HIV/AIDS, malaria, and other priority diseases; (vi) initiate a system to control newer health threats and protect health risk by improving emergency services; and (vii) improve the prevention and control of non-communicable diseases.

The Project Appraisal Document (PAD, pp. 4-5) states that the project “assists the Government of Bangladesh in the implementation of its Strategic Investment Plan (SIP), 2003-2010,” and that “the main purpose of SIP (2003-2010) will be to increase availability and utilization of user-centered, effective, efficient, equitable, affordable, and accessible quality services, be it the Essential Services Package, improved hospital services, nutritional services, or other selected services.” The next sentence refers to these as “objectives,” and these are also listed as the objectives in the PAD’s data sheet and results framework (p. 33).

Since the objectives statements in the DCA and in the PAD are completely different, this Review will assess achievement of the objectives as stated in the DCA. The objectives as stated in the PAD will be used explicitly as part of the causal chain leading to the results envisioned in the DCA's objectives.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project contained three components, intended to constitute a specific set of activities within the Government’s overall investment plan for the sector:

1. Accelerating Achievement of Health, Nutrition, and Population (HNP)-Related Millennium Development Goals (MDGs) and Poverty Reduction Strategy Paper Goals (appraisal, US$ N/A; actual, US$ N/A). This component was intended to support the delivery of a package of essential services. The Essential Services Delivery (ESD) was to focus on reduction of maternal mortality; reduction of neonatal mortality; reduction in childhood morbidity and mortality; improvement in nutritional status, particularly of adolescent girls, pregnant and lactating women, and children; reducing fertility to replacement level; and reducing the burden of tuberculosis and malaria and preventing and controlling HIV/AIDS. Activities were to include a very broad array of public information campaigns, social mobilization and counseling, training of providers, provision of equipment, drugs, and laboratory supplies, provision of immunizations and micronutrients, and voucher programs to increase demand for services.

2. Meeting Emerging HNP Sector Challenges (appraisal, US$ N/A; actual, US$ N/A). This component was intended to support the development of policies and strategies for emerging challenges, with a focus on reduction of injuries and implementing improvements in emergency services; prevention and control of major non-communicable diseases (NCDs); urban health service development; and improvement of the HNP response to disasters. Activities were to include information campaigns, advocacy and counseling programs, legal assistance programs, various kinds of training, establishment of emergency care facilities in high-risk locations, improvements in screening and diagnosis for various conditions, improvements in intersectoral liaison and coordination, and development of a risk management plan.

3. Advancing HNP Sector Modernization (appraisal, US$ N/A; actual US$ N/A). This component was intended to address three key HNP reforms:
a. Public health sector management and stewardship capacity, focusing on improving institutional and personal skills for better planning and monitoring; improved budget management through a medium-term budgetary framework process; reform management; improved aid management; development of proper contract documents and management of contracts with private and non-government organization (NGO) providers; information management; and development of alternative financing mechanisms. Also part of this component was a step-wise delegation of responsibility to promote decentralization and local-level planning.
b. Health sector diversification, through building Ministry of Health and Family Welfare (MOHFW) capability to become an active service purchaser in partnership with NGOs and private providers. The pattern of service provision was to be adjusted over time, with increasing use of contracts and commissions for NGOs to provide primary and secondary care in areas where they had a comparative advantage, and for private providers to provide secondary and tertiary services for poor people where they could do so cost-effectively and with high quality.
c. Stimulation of demand for HNP services, by improving the sector’s image and giving greater attention to effective communication, education, and information strategies for key health programs, and expanding demand-side financing, including piloting of several demand-side financing schemes.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: Neither estimated nor actual costs are provided by component. Total project cost at appraisal was estimated at US$ 4.3 billion.

Financing: The project was intended to finance a specific set of activities within the Government’s overall sector investment plan through a Sector-Wide Approach (SWAp), with the Bank and other partners (United States Agency for International Development, Asian Development Bank, Canadian International Development Agency, United Kingdom Department for International Development, the European Commission, the Government of Germany, the Government of Japan, the Netherlands Ministry of Foreign Affairs, the Swedish International Development Cooperation Agency, the United Nations Children’s Fund, the United Nations Fund for Population Activities, and the World Health Organization) contributing to implementation of the Government’s 2003-2010 Health, Nutrition, and Population Strategic Investment Plan. The Bank was to finance US$ 300 million of total anticipated plan costs of US$ 4.306 billion over the 2003-2010 time period. The Bank’s contribution was pooled with funds from seven of these partners into a multi-donor trust fund (MDTF), which the Bank administered. The total amount of this MDTF, including the Bank’s US$ 300 million contribution, was planned at US$ 687.9 million. Non-pooled partner contributions were planned at US$ 512 million. The actual disbursed amount was US$ 293.4 million from the Bank credit and US$ 387.8 million from the other seven contributors to the MDTF. The ICR does not provide actual total costs for implementation of the overall sector investment plan.

Borrower Contribution: As this was a SWAp, the Government contributed to the overall sector investment plan. The Government’s expected contribution over the 2004-2010 time period to the overall sector investment plan was US$ 2.726 billion (PAD, p. 62). The ICR does not provide the Government’s actual contribution to the plan. The project team subsequently clarified that the Government's actual contribution was US$ 3.1 billion.

Dates:

On October 22, 2009, the project was restructured to modify several indicators, reallocate some funds from program activities that were progressing inadequately to increased pro-poor activities, and to extend the project’s closing date by one year from December 31, 2010 to December 31, 2011 to ensure full use of remaining project funds.

On July 30, 2010, the amounts to be contributed by the Canadian International Development Agency and the Government of Germany were increased, and the amount from the UK Department for International Development decreased.

On March 2, 2011, funds were reallocated between disbursement categories.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Relevance of Objectives is rated Substantial. At the time of appraisal, key issues faced by the sector included a wide gap in health conditions between rich and poor, inadequate reliance of the population on the public health sector (with most spending out-of-pocket, and on private and/or traditional sources of care), poor regulation of quality of care, serious governance issues in the sector, changing epidemiology with increased incidence of injuries, accidents, HIV/AIDS, Hepatitis B and C, cancer, and cardiovascular disease, and poor service to marginalized groups (especially poor women and children from tribal populations, people with disabilities, the elderly, adolescents, and HIV/AIDS patients). The Government’s Sector Investment Plan and the objectives of the project were substantially relevant to these issues. The project remains substantially relevant to Bank strategy, with the current 2011-2014 Country Assistance Strategy stressing inclusive growth and stronger governance through improved social service delivery. Sector-wide approaches with other partners to improve delivery of health, nutrition, and population services are explicitly endorsed in this Strategy (p. ii). The statement of objectives in the financing agreement was completely different from that in the PAD.

b. Relevance of Design:
Relevance of Design is rated Modest.The ICR (p. 7) notes that, while the choice of a SWAp that financed a specific set of activities was appropriate, a broader SWAp (financing a time slice of the entire investment program, rather than financing a specific set of activities) may have laid a better foundation for more far-reaching sector dialogue. Indeed, the choice of such a hybrid approach between a project and a sector-wide program led to serious implementation difficulties and delays resulting from the need to apply the SIM's fiduciary requirements to a very large investment program and the Government's weak capacity in that regard. The use of the MDTF mechanism to pool funds was an “efficient and practical” way to align funding from a large number of development partners (ICR, p. 16).The PAD (pp. 33-51) contains a detailed results framework linking the project’s activities to expected outcomes. However, during implementation significant shortcomings in the design of these activities became apparent. The project’s Program Implementation Plan contained 38 Operational Plans that were poorly integrated. The activities under the third component designed to shift the Ministry of Health and Family Welfare from being a provider to purchaser of health services weighed heavily on the scarce managerial resources of the Ministry, and were made even more challenging by political transitions that took place in 2006 and 2009. Human resource constraints (borne largely of a structural divide between the Revenue and Development parts of the government budget) were not taken adequately into account during project design, resulting in failure to create necessary posts, frequent unfilled vacancies, and ultimately newly built facilities failing to provide planned services due to shortages of needed manpower.


4. Achievement of Objectives (Efficacy) :

Attribution: The pooled funds to which the Bank contributed accounted for only part of the overall sector program, and therefore the observed outcomes relied on a number of factors both within and outside the pooled fund. These include funding from other sources (Government revenue, plus funding from other development partners outside the pool), as well as non-health sector factors that contribute to achievement of many of the project’s objectives. In terms of health sector funding, the ICR (p. 18) points out that the majority of the MOHFW’s revenue funding was allocated to salaries, with little flexibility in the short term, and therefore the contribution of the pooled funding appears to have been substantial in terms of the proportion of key inputs it financed (new initiatives, drugs, etc.). However, some key development budget interventions of the MOHFW were outside the purview of the project, including a project on revitalization of community clinics (ICR, p. 41). According to the main text of the ICR, it is reasonable “to assume that the HNPSP has contributed significantly” to observed outcomes in the sector (ICR, p. 18), within the context of the Bank’s and the pooled funding’s overall support to the Government’s program; but comments on the ICR by the Ministry of Planning’s Implementation, Monitoring & Evaluation Division (IMED, ICR, p. 41) state that “in view of its limited boundary, it will be unrealistic to assign the national health outcomes to effects of the HNPSP.”

Other elements of the ICR raise caveats about the quality of data available to assess progress. According to comments by one development partner, there is reason to be concerned that reported data come largely from the Government’s own systems, and “the lack of third party verification or triangulation of data and results continues to be a concern deepened by the recognition of overall ‘weak internal controls’ and limited progress in monitoring and evaluation in the sector” (ICR, pp. 25, 50). Also, although precise data are not provided, one development partner cites “considerable private sector health provision in Bangladesh, some of this by public providers themselves,” raising questions about the extent to which observed results can be attributed to the public sector through the SWAp (ICR, pp. 25, 50). The project team subsequently clarified that the majority of the data on project outcomes are from Demographic and Health Surveys (DHS) and the Multiple Indicator Cluster Survey, and that the Government takes its information from those surveys. Additional data for this Review has been added from the 2011 DHS.

One development partner also points out that the revision of indicators and targets in 2009 translated into “the lowering of expectations concerning the program results. If one had not done this, the assessment would have been not so good” (ICR, p. 49).

Outputs Relevant to All or Most Objectives:

The MOHFW finalized a National Health Policy that was aligned with the goals established by the Second National Strategy for Accelerated Poverty Reduction. A regulatory framework for pharmaceuticals was established. Overall, according to the ICR, “there has been limited progress in implementation” of health sector modernization, “due in part to the setting of over ambitious expectations that did not sufficiently take into account political economy considerations” (p. 29).

The Ministry of Local Government, Rural Development and Cooperatives (MOLGRDC) contracted with NGOs to implement health services in urban areas. Under these arrangements, MOHFW provides required commodities such as contraceptives and TB drugs. MOLGRDC is currently in the final stages of development of a strategy, in coordination with MOHFW, for urban health services.

Overall, the ICR provides little detail on the project’s specific outputs, making it difficult to construct a plausible causal chain linking the project’s activities to observed outcomes. The project team subsequently clarified that the project financed a slice of the Government's overall sector program, making it impossible to disaggregate project-specific outputs from the entire range of health sector activity during the time period of the project.

Outputs related to the availability and utilization of effective, user-centered services: The percentage of total government expenditures allocated to the MOHFW increased from 6.5% in 2004 to 6.7% in 2007/2008, not meeting the target of 10%. There is still chronic underspending of the MOHFW development budget, even though the program took “several steps” (not specified) to expedite fund release (ICR, p. 32). The unspent portion of the development budget fluctuated during the project period: 27.6% in 2003/2004; 17.2% in 2004/2005; 13.6% in 2005/2006; 25.2% in 2006/2007; 17.1% in 2007/2008; and 26% in 2008/2009. According to an evaluation performed by the Ministry of Planning’s Implementation, Monitoring & Evaluation Division (IMED), 80% of 490 patients surveyed reported that services provided at government health facilities had improved over time. The same IMED evaluation conducted focus groups with patients, members of local government bodies, private physicians, teachers, community youth leaders, and farmers. These focus groups, according to the ICR (p. 39), indicated that the project’s interventions had resulted in “marked improvement” in the quality of health services provided through Upazila Health Complexes, the availability of doctors, nurses, and equipment, and the supply of essential drugs to patients (especially to women, children, and the poor). However, these focus groups also reported shortcomings in equipment available to health workers and supervision at health facilities.

Outputs related to the availability and utilization of efficient services: According to the ICR (p. 28), major policy decisions on decentralization and hospital autonomy were not reached due to the changing political situation. Pilots on management and autonomy were carried out in six district hospitals and 14 Upazila Health Centers. Pilot local-level planning was carried out in six districts and their Upazilas, and 2009 budgets reflected those pilots. The percentage of MOHFW expenditure on medical and surgical requisites at the district level and below increased from 9% in 2004 to 67% in 2006/07 (there was no target). The percentage of MOHFW expenditure at the Upazila level and below declined from 51% in 2004 to 47% in 2008/09, not meeting the target of greater than 50%.

Outputs related to the availability and utilization of equitable, affordable, and accessible services: The percentage of total MOHFW expenditure allocated to the 25% poorest districts was 15% in 2006/2007. No baseline or more recent data are available. The target of 40% appears not to have been achieved. The focus groups conducted as part of the IMED evaluation reported shortcomings in access to health care for the poor, specifically, that “there should be more attempts to take health services to the door-steps of poor people” (ICR, p. 39).

(i) Reduce infant, under-five, and maternal mortality and the proportion of malnourished children, and eliminate the gender disparity in child nutrition and mortality: Modest achievement. Targets set under this objective were derived from the Millennium Development Goals, adjusted to the project's timeline. Many targets were not met, although there was progress toward targets on almost all indicators, with reason to expect that this progress will continue. It is also important to note that observed outcomes are likely attributable to many factors beyond the scope of the project.

Outputs

The percentage of children under one year of age who were fully immunized increased from 73% in 2003 to 80% in 2011, not meeting the target of 85%. The 2011 DHS reports that 83% of children age 12-23 months had been fully immunized by their first birthday. The percentage of newborns protected at birth against tetanus increased from 86% in 2003 to 91.3% in 2010, not meeting the target of 95%. The 2011 DHS states that 89.9% of newborns were protected at birth against tetanus.

The percentage of children age 1-5 receiving Vitamin A supplements in the previous six months increased from 81.8% in 2004 to 92.2% in 2011, exceeding the target of 90%. There was a less than 10% difference in progress between the poorest and richest wealth quintiles between 2004 and 2010 in vitamin A supplementation (for children 9-59 months) and full immunization coverage (for children under 2).

The percentage of children under 5 with symptoms of acute respiratory infection who sought care from a trained provider increased from 74.6% in 2004 to 82.7% in 2010. No target was specified. The 2011 DHS reports that the percentage of children under five with symptoms of acute respiratory infection in the preceding two weeks who had sought care from a health facility or health provider decreased from 37% in 2007 to 35% in 2011.

A demand-side financing pilot (a maternal voucher scheme for pregnant women) was implemented in 33 Upazilas. In 2010, 64% of pregnant women who were targeted by a voucher scheme delivered their babies using a skilled birth attendant, exceeding the target of 60%.

The percentage of births attended by skilled personnel increased from 13.4% in 2004 to 26.5% in 2011, not meeting the target of 50% (or a 2009 revised project target of 28%). The 2011 DHS reports that this percentage increased from 16% in 2004, to 21% in 2007, to 32% in 2011. The percentage of antenatal care provided by medically trained providers increased from 48.7% in 2004 to 56.0% in 2010, not meeting the target of 75%. The 2011 DHS reports that the percentage of live births in the preceding three years for which women received at least one antenatal care visit from a medically trained provider increased from 50.5% in 2004, to 53.4% in 2007, to 54.6% in 2011.

The National Nutrition Program area-based community nutrition program was expanded to 63 new Upazilas.

Outcomes

The maternal mortality rate declined from 322/100,000 in 2001 to 194/100,000 in 2010, exceeding the target of 240/10,000.

The neonatal mortality rate declined from 41/1000 in 2004 to 32/1000 in 2010, nearly meeting the target of 30/1000. The 2011 DHS also reports neonatal mortality at 32/1000.

The under-five mortality rate declined from 88/1000 in 2004 to 56/1000 in 2010, nearly meeting the target of 52/1000. The 2011 DHS reports under-five mortality at 53/1000, essentially reaching the target.

The infant mortality rate declined from 65/1000 in 2004 to 45/1000 in 2010, not meeting the target of 37/1000. The 2011 DHS reports infant mortality at 43/1000, not meeting the target.

The percentage of children under five (age 6-59 months) who were underweight declined from 47.5% in 2004 to 38.6% in 2010, not meeting the target of 34%. The 2011 DHS reports that 36% of all children under five were underweight, more closely approaching the target. The percentage of children under five (age 24-59 months) whose growth was stunted declined from 43.0% in 2004 to 35.5% in 2010, not meeting the target of 30%. The 2011 DHS reports that 41% of all children under five were stunted, again not meeting the target.

The ICR does not provide data related to gender disparities. The project team subsequently provided the following additional tabular data, demonstrating a narrowing of the gap between boys and girls from 2004-2007 along most indicators, and for Vitamin A supplementation, girls surpassing boys in 2006 and 2010. The 2011 DHS indicates that the infant mortality rate for girls in was 37/1000, and that for boys 43/1000, further narrowing the gap. The under-five mortality rate as reported by the 2011 DHS, however, was 50/1000 for girls and 57/1000 for boys, representing a widening of the gap between 2007 and 2011. The 2011 DHS reports that 59% of boys and 60% of girls age 6-59 months were given Vitamin A supplementation in the preceding six months, essentially indicating gender parity. The 2011 DHS gives figures of 42% for females and 41% for males for stunting under age five, and 39% for females and 34% for males under age five for underweight. This represents a maintenance of gender parity for stunting, but an increase in the gap between boys and girls for underweight.







(ii) Increase access to reproductive health services: Substantial achievement.

Outputs

The ICR reports no outputs related to this objective. The project team subsequently added that the government implemented programs to address and strengthen local-level planning, including family planning.

Outcomes

The percentage of births attended by skilled personnel among the lowest two wealth quintiles increased from 3.3% in 2004 to 11.8% in 2010, exceeding the target of 10%.

The percentage of antenatal care provided by a medically trained provider for the lowest two wealth quintiles increased from 24.9% in 2004 to 40.3% in 2010, meeting the target of 40%.

The contraceptive prevalence rate (modern methods) increased from 47.3% in 2004 to 54.1% in 2010, not meeting the target of 60%. The 2011 DHS indicates that use of modern methods increased from 47.3% in 2004, to 47.5% in 2007, to 52.1% in 2011, approaching but again not meeting the target. The percentage of eligible couples or women on long-lasting birth control methods remained essentially stable, at 7.2% in 2004 and 7.4 in 2010, not meeting the target of 9.3%. The 2011 DHS reports that 11.2% of women used injectables and 0.7% used intrauterine devices, surpassing the target.

(iii) Lower fertility with a view toward achieving replacement-level fertility by 2010: Substantial achievement. It is important to note that other factors, such as increasing age of marriage, were likely contributors to observed outcomes.

Outcomes

The total fertility rate (TFR) declined from 3.0 children per woman in 2004 to 2.5 in 2010, not meeting the target of 2.2. Replacement-level fertility was not achieved. However, the 2011 DHS gives the TFR as 2.3, essentially meeting the target.

(iv) Reduce the burden of TB, HIV/AIDS, malaria, and other priority diseases: Substantial achievement.

Outputs

The TB case detection rate increased from 46% in 2004 to 74% in 2010, essentially meeting the target of 75%.

The percentage of districts with disease surveillance reports increased from 52% in 2004 to 95% in 2011, meeting the target.

100% of districts were using Directly Observed Treatment Short-Course (DOTS) by project closure.

Outcomes

The TB cure rate increased from 85% in 2004 to 92% in 2010. The ICR gives targets of 85% (Data Sheet) and 95% (p. 54).

Although no outputs on malaria were reported, the ICR's economic analysis includes data that the malaria death rate per 1000 declined from .0053 in 2003 to .0034 in 2010.

The project team later explained that an HIV epidemic feared possible at the time of preparation did not materialize, and therefore no HIV-related outcomes were monitored or reported.

(v) Initiate a system to control newer health threats and protect health risk by improving emergency services: Modest achievement.

Standard operating procedures were put in place for mass casualty management and disaster mitigation for field-level health personnel. However, according to the ICR (p. 28), “the quality of any response may be hampered by the often fairly low level of institutional capacity and service quality in the health sector.”

Four one-stop crisis centers were established in four divisional Medical College Hospitals to address violence against women. The manual that guides management of violence was updated in 2007, and training in this area was initiated for health service personnel. According to the ICR (p. 28), there are “limitations regarding appropriately trained personnel which constrain the availability of some services.”

(vi) Improve the prevention and control of non-communicable diseases: Modest achievement.

Two strategies were developed and endorsed: a Strategic Plan for Surveillance and Prevention of NCDs 2011-2016, and a National Cancer Control Strategy and Plan of Action 2009-2015. An NCD risk behavior survey was conducted.

Tobacco usage among men and women over age 15 increased from 20.9% in 2004 to 23% in 2009 for smoking tobacco, and from 19.7% in 2004 to 27.2% in 2009 for smokeless tobacco, not meeting the target of 15% for both smoking and smokeless tobacco.

According to the ICR (p. 28), progress in this area was “limited.”


5. Efficiency:

Efficiency is rated Modest.


The PAD (pp. 97-108) calculated a Net Present Value of US$ 610 million (5 year) and US$ 2.975 billion (10 year) and an Internal Rate of Return of 21% and 51% for 5 and 10 year horizons, based on a set of assumptions that were largely achieved during implementation (though data are not available for many of the assumptions) (ICR, pp. 36-37). The ICR did not repeat the calculations at closing, and therefore little data are available on the efficiency of investment of Bank resources, resulting in a Modest rating.

The project’s primary health care interventions, including immunization, antenatal care, increasing access to skilled birth attendants, etc., were among those generally considered to be cost-effective. The project’s poverty focus successfully directed funding toward those most in need (ICR, p. 18). Although overall per capita financing of the public sector for health is low (US$ 4-5/person/year), a 2010 Bangladesh Public Expenditure and Institutional Review found that high value is obtained from those expenditures compared to other countries in the region (ICR, p. 19).

The 38 “silo-like” Operational Plans were identified as an important problem by the 2008 mid-term review, producing inefficiencies such as constructed facilities sitting idle because of poorly coordinated procurement of equipment/drugs and allocation of human resources (ICR, p. 9). A 2008 Bank survey of procured equipment found “considerable wastage of project resources due to uncoordinated central procurement” (ICR, p. 46). Throughout the project, expenditures were higher in the last quarter than in the first two quarters, likely caused by delays in release of funds to cost centers (ICR, p. 33). Furthermore, the bifurcation of services between the Directorate of Health and the Directorate of Family Planning, from the center down to the lowest service delivery points at the field level, led to duplication and avoidable waste of financial and human resources (ICR, p. 44). The mid-term review commented on the negative effects of this bifurcation and recommended improved coordination, but the Ministry of Health and Family Welfare continued to support the division. The project team subsequently stressed that these structures are an unavoidable part of the government's operational arrangements, and that the team worked consistently with the Bank and the government to mitigate these challenges in the health and other sectors. These efforts included development of a single work plan and harmonization of the development and revenue budgets.


a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project’s outcome is rated Moderately Satisfactory, based on Substantial relevance of objectives, Modest relevance of design, Substantial achievement of three objectives, Modest achievement of three objectives, and Modest efficiency. The project’s objectives remain substantially relevant to country conditions, Government strategy, and Bank strategy, but its design failed to take into account considerable capacity and institutional constraints. The project appears to have made substantial gains in the diagnosis and treatment of tuberculosis. Although targets were not reached on several of the Millennium Development Goals that were integral to the project, important progress was made, with the reasonable expectation that these targets will be reached. There is little information available on the cost-effectiveness with which Bank resources were invested.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

A follow-on Health Sector Development Program (US$ 358.9 million, 2011-2018) has ensured continuing support, including components for improving health programs and service provision and strengthening health systems. Facilities that were constructed under the project, including comprehensive emergency obstetric care centers, maternal and child welfare centers, and other facilities serving the poor at the Upazila level, remain fully functional, though improvement in management capacity did not take place in tandem with facility construction and upgrades and remains a risk to further development of the sector (ICR, p. 19). General under-financing of the sector constitutes an additional risk, though the ICR (p. 35) notes that, given economic growth of 6%, Bangladesh could increase the amount it allocates to health should it consider the health sector a political priority. The ICR does not assess the likelihood that this prioritization has taken place or will take place. One development partner’s comments (ICR, p. 50) indicate that, while the health budget has increased by 10% annually since 2008/2009, inflation has averaged 10.5%, resulting in a decline in real terms.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
Preparation specifically incorporated lessons learned from previous interventions in the Bangladesh health sector (PAD, p. 8), including clear identification of the role of development partners and the Government, alignment of technical assistance with policy documents, careful consideration of the political economy of reform, and the importance of matching demand-side strategies to supply-side interventions. Preparation involved all major stakeholders, including development partners, civil society organizations, and the Bangladesh Medical Association. The Bank led the process of alignment of a large number of development partners.


Risks were adequately identified (PAD, pp. 13-14), though the risk of “commitment for the proposed sectoral reforms” was rated too low, and mitigation strategies for governance and capacity issues proved inadequate. Key actors in the Government outside the Ministry of Health and Family Welfare were not fully on board with the planned doubling the share of the Government budget going to the Ministry. Underlying systemic problems in the sector, particularly those related to human resources and decentralization issues, were not properly identified and dealt with, impeding the efficient delivery of services throughout the life of the project; these constraints, though falling outside the boundaries of the health sector, could have been better anticipated in the design stage (ICR, p. 16). Capacity issues in the Ministry were not adequately addressed.

Project design was complex, with many financiers, three components, and five disbursement categories. The 38 Operational Plans, which were the main instruments for the content and implementation of the project, were centrally planned, without any input from or interaction with field-level implementers. The OPs were “hastily prepared, processed, and approved,” with each OP a self-sufficient and stand-alone unit unconnected to the others; this led to significant implementation challenges and inefficiencies. Although these OPs were an integral part of the government's organizational framework, it appears that there was insufficient effort at coordination during preparation.

The objectives of the project (supported by the pooled fund) were the same as those for the overall sector investment plan, though the descriptions of the objectives in the PAD and the legal agreement were different. The indicators chosen to measure progress on these objectives were too narrowly focused, and some were difficult to measure in practice, leading to changes in the indicators during the 2009 restructuring.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
The Bank team identified problems and restructured the project in 2009, following recommendations from the 2008 mid-term review, but without aligning the different statements of the project's objectives. Turnover of Task Team Leaders was high (four TTLs during implementation), but continuity was strong, with two of the TTLs resident in Bangladesh and Bank staff very engaged with project monitoring and support. The Bank team provided regular and appropriate updates to pool funders and other development partners, managing a heavy workload with the pooled funding arrangement and large number of Operational Plans. Regular visits were made to field facilities and program areas to oversee activities implemented by non-governmental organizations and civil works sites.

However, the 2009 restructuring did not go far enough, given remaining problems with capacity constraints and coordination between the 38 Operational Plans. The project did not provide comprehensive support to improve the Ministry of Health and Family Welfare’s capacity in procurement and financial management (ICR, p. 14). The mid-term review in 2008 recognized a mismatch between project design and available capacity, suggesting (for this and other reasons) a redesign that would emphasize service delivery and a strengthened support system (including human resources, procurement, monitoring and evaluation, and financial management), but this suggestion was not acted upon, and therefore Ministry capacity constraints affected aspects of project implementation until closing (ICR, p. 43). Implementation was largely top-down, “without being responsive to the needs or suggestions originating from the actual implementers” of the project’s activities (ICR, p. 45). There was limited scope for learning from experience. One development partner commented that there was room for strengthening of coordination between development partners and the government, and among development partners, in order to increase alignment and harmonization (ICR, p. 25).

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Bangladesh had several changes of government during the implementation period (2006, 2008, and 2009), leading to changes in national priorities and transfer of staff in key positions. Overall, however, the health sector remained a priority for each Government throughout the project implementation period. Although a decentralization policy was approved in 2009, it was not implemented and the health system in practice remained highly centralized, leading to challenges in improving health service delivery. Integrated budgeting and planning in the sector was highly impeded by the division of the Ministry of Health and Family Welfare into separate directorates for health and family planning, and further separation of planning for development and revenue budgets, with the revenue budget allocated to facilities/institutions and the development budget allocated to programs. Together with the Program Implementation Plan’s separation of implementation responsibility among 38 silo-like Operational Plans, this Government structure created gross inefficiencies due to lack of synchronization and coordination.Necessary action to develop a system for national placement and retention of key human resources, particularly health service providers, did not take place despite strong recognition of the need to do so. Planned diversification through contracting of non-governmental organizations also did not take place, as the new government in 2006 did not want to contract out essential health services; however, eventually contracting did take place with NGOs in the areas of nutrition, HIV, and tuberculosis..

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
The Ministry of Health and Family Welfare was the implementing agency. Its activity took place under severe financing and capacity constraints, and within a context of attempted influence by many different political and other interest groups. In other words, the environment for project implementation was very complex. Creation of a Project Support Office and Management Support Agency took place in 2007 and 2008 respectively, but these agencies were never fully connected to the Ministry, particularly after the first change of government in 2009; they were never fully functional, and their contracts were discontinued in 2010. There was never a full understanding of their respective roles, and they were seen as disconnected from day-to-day implementation. A planned Performance Monitoring Agency never materialized. Line Directors did not have the necessary training, exposure, or institutional support to play their roles effectively as planners and managers of implementation, and there was high turnover of personnel in these positions. The Ministry did not take ownership of the annual performance review process, and it took no follow-up action on recommendations from these annual reviews (ICR, p. 48).

There were substantial deficiencies in procurement planning, processing, and tracking, resulting in considerable wastage of project resources due to lack of coordination (ICR, p. 46). These weaknesses are presented in Section 11.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The project established a Monitoring and Evaluation Unit in the Ministry, with plans to strengthen the overall health Management Information System and make full use of the Bangladesh Demographic and Health Survey and Utilization of Essential Services Delivery Survey. The PAD (pp. 33-51) lists two sets of indicators for the project: one with 62 indicators covering the full sector program (revised after the mid-term review to 30 indicators), and another for the pooled fund, with ten key project indicators. Some of these indicators proved impossible to monitor and were later revised. Institutional responsibility for data collection and analysis, along with a timeline, are established in the PAD.

b. M&E Implementation:
Progress was guided by a series of independent Annual Program Reviews, consisting of two parts: an assessment by an independent team, and then an Action Plan jointly agreed to by the Government and development partners, based on that independent team’s assessment. Some recommendations by the independent teams were not implemented due to constraints in Government systems. Project indicators were regularly monitored and reported throughout implementation, but there were problems with coverage, data quality, and fragmentation. The Monitoring and Evaluation Unit was established but never fully staffed, and challenges were presented by bifurcation of routine data systems between the Directorate of Health Services, the Directorate of Family Planning, and private sector providers. Development partner comments state that“the lack of third party verification or triangulation of data and results continues to be a concern deepened by the recognition of overall ‘weak internal controls’ and limited progress in monitoring and evaluation in the sector” (ICR, p. 50).

a. M&E Utilization:
The Annual Program Reviews were the main source of dynamism in performance evaluation and establishing the future course of action for the project and the overall Government sector investment program (ICR, p. 8), although those reviews themselves pointed to a low level of use of data for decision making at all levels (ICR, p. 13). Monitoring was restricted to one-way reporting from the field to higher levels, with little systematic feedback from those levels or from the line directors or the directorates of health services and family planning (ICR, p. 46). Overall, according to the Summary of the Borrower’s ICR (p. 46), “monitoring and supervision remained a neglected activity in the project.”

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The project triggered OP/BP/GP 4.01, Environmental Assessment, and OD 4.20, Indigenous Peoples.The project complied with all safeguard requirements and policies (ICR, p. 13). Neither the PAD nor the ICR state the environmental screening category of the project. The project team subsequently clarified that the project was Environmental Category C. A Health Care Waste Management Guideline was developed, and the Directorate of Health Services took the lead in implementing this plan at health facilities. At the design stage, it was not envisioned that the project would finance civil works, but following the mid-term review in 2008, it was agreed that the project would finance some civil works in poor areas. The Bank worked with the Government to implement an Environmental Management Plan that has been prepared at appraisal, ensuring successfully that civil works financed from the pooled funds complied with social safeguards and environmental policies of the Bank. There were no adverse impacts of the project on indigenous people (ICR, p. 13).

b. Fiduciary Compliance:
Quarterly financial management reports were submitted mostly on time. The Ministry of Health and Family Welfare was the first in the country to outsource the internal audit function. Between fiscal years 2006 and 2011, the Bank identified 36 audit observations (worth US$ 68.14 million) as material and substantive from the observations raised by the Foreign Aided Projects Audit Directorate auditors. Although they have all been resolved, the pace of doing so was slow (ICR, p. 14). The Ministry of Health and Family Welfare organized regular Financial Management Task Group meetings to provide an overview of progress on sectoral financial management. There were weaknesses during implementation related to the weak internal controls environment and high staff turnover. Mitigation measures were put in place during preparation and during implementation, but these did not aim at long-term capacity building.

Of five central procurement entities, two struggled to prepare quality procurement documents and therefore required considerable support. Modest progress was noted in strengthening procurement quality in these two agencies. Challenges remained, however, including high staff turnover, lack of capacity in dealing with complaints, and delays in conducting procurement audits. There was a high level of waste of project resources due to uncoordinated central procurement, involving many instances where equipment or drugs were allocated to places or institutions that did not need them (or because the required technical staff or resources for their operation were not made available).

Overall, the project's fiduciary requirements placed a large burden on the Government's relatively weak financial management and procurement capacity, which the project did not sufficiently strengthen.

c. Unintended Impacts (positive or negative):
None reported in the ICR.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
The project’s outcome is rated Moderately Satisfactory, based on Substantial relevance of objectives, Modest relevance of design, Substantial achievement of three objectives, Modest achievement of three objectives, and Modest efficiency. The project’s objectives remain substantially relevant to country conditions, Government strategy, and Bank strategy, but its design failed to take into account considerable capacity and institutional constraints. The project appears to have made substantial gains in the diagnosis and treatment of tuberculosis. Although targets were not reached on several of the Millennium Development Goals that were integral to the project, important progress was made, with the reasonable expectation that these targets will be reached. There is little information available on the cost-effectiveness with which Bank resources were invested. 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Satisfactory
Moderately Satisfactory
Key risks were not adequately assessed and mitigated during preparation, including underlying systemic problems in the sector related to human resources and institutional coordination. The lack of integration of 38 separate Operational Plans to guide implementation resulted in considerable implementation challenges and inefficiencies.  
Borrower Performance:
Satisfactory
Moderately Unsatisfactory
Institutional complexity in the Government and the Ministry of Health and Family Welfare created significant unresolved challenges and inefficiencies. Necessary action on institutional coordination and human resource challenges did not take place. The Implementing Agency operated under capacity constraints. While the Ministry was operating in a challenging political context, it did not assume full ownership of the challenges the project faced during implementation. 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons are adapted from the ICR (p. 23):

The results framework should include indicators that are directly linked to the project, within the purview of the implementing agency, and measurable. In this case, there were too many indicators, and many of those were affected by a range of factors outside the scope of the project. As a result, it is difficult to draw a results chain directly from the project’s activities to its outcomes and impact measured by the project’s formal indicators.

Project preparation must adequately take into account a range of political economy variables, including the possibility of changes in government. Flexibility should be incorporated into project design from the outset, in order to cope nimbly with changes in government priorities.

This lesson is offered by IEG:

Capacity building needs must be clearly and fully identified up front, and dealt with early in the project implementation period. In this case, capacity constraints at virtually all levels significantly hampered implementation progress.


14. Assessment Recommended?

Yes
Why?

To learn lessons from the multiple political economy factors that influenced project design and implementation, and to verify the ratings.

15. Comments on Quality of ICR:

The ICR is clear and concise, with a strong discussion of the challenges of attributing observed outcomes to the project’s interventions (p. 18). However, there were important shortcomings. The ICR provides little detail on the project’s outputs, making it difficult to construct a plausible causal chain linking the project’s activities to observed outcomes. Project cost data are incomplete. The content is primarily descriptive rather than analytical. The project team had to provide a large amount of supplementary information to support this Review. The Summary of the Borrower’s ICR, based on an evaluation of the project by the Ministry of Planning’s Implementation, Monitoring, and Evaluation Division, as well as the comments in the ICR from several development partners, are considerably more candid about the project’s challenges and response to those challenges than is the main text of the ICR.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions