|1. Project Data:
ICR Review Date Posted:
|Sv - Earthquake Emergency Recovery & Health Services Extension Project
Project Costs(US $M)
Loan/Credit (US $M)
|Health, Nutrition and Population
Cofinancing (US $M)
Board Approval Date
|Health (95%), Central government administration (5%)|
|Health system performance (25% - P)
Natural disaster management (25% - P)
Decentralization (24% - P)
Population and reproductive health (13% - S)
Nutrition and food security (13% - S)|
||ICR Review Coordinator:
||Judyth L. Twigg
|2. Project Objectives and Components:|
a. Objectives:According to the Project Appraisal Document (PAD, p. 2) the development objectives were:
" to restore hospital operations and minimize losses in health status to vulnerable populations living in the country's earthquake damaged central and para Central Regions and to improve the health status of underserved populations elsewhere, with special emphasis on the poverty-stricken Northern region."
According to the Loan Agreement (Schedule 2), the objectives of the project were:
"(a) to reconstruct and improve the borrower's health sector infrastructure damaged or destroyed by the earthquakes; (b) to extend the coverage of the borrower's health and nutrition services; and (c) to strengthen the institutional capacity of Ministry Of Health (MOH) to develop and implement policies and priority programs for the health sector."
This review will use the PAD’s statement, as it points to the outcomes that were intended to be achieved by the output-oriented statement in the lending agreement.
b. Were the project objectives/key associated outcome targets revised during implementation?
c. Components:Component I: "Emergency reconstruction of MOH hospital network in earthquake-affected areas" (appraisal US$ 127.0 million, actual US$ 137.97 million)
This component aimed at: (1) rehabilitating/replacing seven of the largest hospitals damaged by the earthquake; and (2) preventive maintenance through strengthening the MOH's integrated preventive maintenance program and a preventive maintenance program for the hospitals being reconstructed under this component. The three hospitals for rehabilitation were San Juan de Dios in San Miguel, San Pedro in Usulután, and Santa Teresa in Zacatecoluca. The four hospitals to be reconstructed were Maternidad Nacional in San Salvador, Santa Gertrudis in San Vicente, Cojutepeque in Cuscatlán, and San Rafael Hospital in La Libertad. In addition, this component aimed to support the unit within the MOH (Infrastructure Executing Unit) that would oversee the implementation of this component, as well as quarterly technical and financial audits.
Component II: "Strengthening essential health and nutrition services in earthquake-affected and extremely poor areas" (appraisal US$ 16.5 million, actual US$ 19.66 million)
This component aimed to: (a) extend essential health and nutrition services to the Northern Zone, where no MOH providers were present, through the development of public-private partnerships; (b) strengthen MOH's primary care delivery in earthquake-affected areas by financing minor equipment, essential drugs, and medical supplies; and (c) strengthen the MOH's capacity to plan coverage extension, manage contracts and performance agreements, and monitor and evaluate performance.
Component III: "MOH institutional development for policy formation, national priority programs, and support systems" (appraisal US$ 16.0 million, actual US$ 10.52 million)
This component sought to strengthen the capacity of the MOH to perform stewardship functions related to quality enhancement, health promotion, public health programs, disease surveillance, regulation, and performance-based monitoring and evaluation. These activities wee to be contingent on the implementation of an institutional strengthening and decentralization strategy to which investments were to be linked. This component had three sub-components: (1) institutional modernization; (2) investments to strengthen MOH national priority programs; and (3) investments to strengthen MOH support systems.
Component IV: "Project management" (appraisal US$ 4.8 million, actual US$ 6.27 million)
This component aimed to: (a) support project management for Components II and III, including the design and implementation of a system to monitor and supervise these two components (monitoring and supervision of Component I was to be financed through Component I); (b) support the impact evaluation of the project as a whole, including Component I; (c) conduct annual financial audits of the project; and (d) finance consultant services, facilities, and equipment for the maintenance of the Project Coordinating Unit (PCU).
d. Comments on Project Cost, Financing, Borrower Contribution, and DatesProject Cost
Actual total project cost was 6.1% higher than anticipated. This was chiefly driven by a large overrun for Component I, where the costs of rehabilitation and reconstruction were underestimated. The delay between original cost estimates and start of construction meant: (1) an increase in the price of construction materials; (2) an introduction of new seismic norms; and (3) an introduction of new operating and functional criteria. Additionally, modifications to hospital designs and lawsuits had adverse costing implications (ICR, p. 26). Furthermore, substantially less was spent on MOH institutional development, and a strategy for human resources (HR) and information technology (IT) was only partially developed and implemented.
The project was financed by an International Bank for Reconstruction and Development (IBRD) loan in the amount of US$ 142.6 million (PAD, p. 12).
Against a planned US$ 23.1 million (14% of total project cost), the Borrower actually contributed US$ 33.5 million (19% of total project cost). This was because extra resources were needed to complete construction of Usulután, Zacatecoluca and San Vicente Hospitals. The Government allocated US$ 11.55 million more to cover legal fees due to arbitrations and settlements (ICR, pp.11, 42).
The Loan Agreement was amended four times to extend the original closing date from 04/30/2007 to 06/30/2011.
The first amendment, in September 2003, reflected the results of negotiations between the Government and the National Assembly. The second amendment, in March 2006, reflected a reallocation of proceeds. The third amendment, in April 2007, extended the project closing date by 18 months to 10/31/2008 so as to give the project enough time to implement the agreed activities. The fourth amendment, in February 2008, involved a third reallocation of proceeds. The fifth amendment, in October 2008, extended the closing date to 10/31/2009. The sixth amendment, in October 2009, both extended the closing date to 03/30/2010 and granted a reallocation of proceeds. The seventh amendment extended the closing date to 10/31/2010, and an eighth one, in October 2010, extended it to 06/30/2011. The ninth and last amendment granted a reallocation of proceeds in May 2011. (ICR, p.56)
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:Substantial. . The development objectives of "restoring hospital operations and minimizing losses in health status to vulnerable populations living in the country's earthquake damaged central and para Central Regions and to improve the health status of underserved populations elsewhere, with special emphasis on the poverty-stricken Northern region" were and remain substantially relevant to country conditions at the time of appraisal, and to Government and Bank strategy. At the time of appraisal one third of the population was estimated to have limited or no access to medical care, with the majority of need concentrated in the country’s Northern Region. Many health indicators were below the Latin America and Caribbean (LAC) region’s average, with particular concern for communicable and infectious disease, malnutrition, diarrhea, and respiratory infections. The Bank's current Country Partnership Strategy (CPS, 2009, current at project closure) contains a focus on improved nutrition and access to basic health care services (pp. 11, 21). A World Bank health sector reform project with Components II-IV was already in the pipeline at the time of the earthquake, and was augmented with Component I as an emergency measure to rehabilitate and reconstruct health infrastructure damaged or destroyed by the earthquakes. The two earthquakes struck in early 2001, killing 1,260 inhabitants. The greatest damage was recorded in the departments of San Vicente, La Paz, Usulután, San Salvador, and Cuscatlán and was estimated to amount to 12 percent of the country’s GDP in 2000. Two MOH hospitals suffered severe damage, and another six hospitals bore sufficient impact to require full or partial evacuation. In total, 113 of 361 facilities were affected, representing 55 percent of MOH’s supply of health care services. The earthquake rendered 2,000 hospital beds out of service, reducing inpatient capacity by 25 percent (PAD, p.4)
b. Relevance of Design:Substantial. The PAD's results chain is plausible. The reconstruction and refurbishment of hospitals in earthquake affected areas credibly feeds into the objective of restoring hospital operations. Similarly, improved health status is conceivably achieved through extending coverage of essential health and nutrition services in the impoverished northern region (through strengthened primary care delivery and provision of essential drugs) and, in the long term, improved institutional capacity.
|4. Achievement of Objectives (Efficacy) :|
To restore hospital operations and minimize losses in health status to vulnerable populations living in the country's earthquake damaged central and para Central Regions: Modest
Six of the seven targeted hospitals were fully reconstructed or refurbished and are fully functioning as planned. Three hospitals (Santa Gertrudis in San Vicente, Cojutepeque in Cuscatlán, and San Rafael in La Libertad) were reconstructed and are fully equipped. Three hospitals (San Juan de Dios in San Miguel, San Pedro in Usulután, and Santa Teresa in Zacatecoluca) were rehabilitated. The planned maternity hospital in Sal Salvador was not built, but the loan financed preparatory studies and some equipment.
All hospital designs were completed, and all hospitals (including the maternity hospital) purchased equipment. A maintenance program was launched in six out of seven reconstructed/refurbished hospitals, meaning that: (a) equipment operators were trained and operating budgets increased; (b) maintenance manuals and trainings were prepared; (c) electrical and heating equipment were upgraded; and (d) generators were installed for medical equipment.
The ICR provides no evidence of the restoration of the 2,000 hospital beds lost during the earthquake or return of hospital inpatient capacity to pre-earthquake levels. The Task Team subsequently added that the project restored 1,366 beds of the 2,000 hospital beds lost during the earthquake, which are fully functional in a more efficient way than before the earthquake as hospital systems were significantly improved. There was thus no need for the additional 612 beds.
Investments were prepared and executed in waste management and disposal, environmental health, and communication strategies. These were only partially achieved in Information Technology and Human Resource Management Systems.
Two functioning HIV testing and counseling centers were set up, meeting the target. A fully functioning web-based information system was developed at MOH, with data on morbidity/mortality rates, HIV surveillance, family records, vectors, and medical supplies.
Data on length of stay were not available for 4 out of the 6 hospitals, since civil works were finished shortly before project closure. The Task Team subsequently added that average length of stay improved slightly to 3.9 days in Cojutepeque and San Rafael Hospitals, which is within the international benchmarks, and that hospital users reported an unspecified reduction of waiting time and improved access to an array of diagnostic tests.
The Task Team also subsequently added that national estimates for infant and neonatal mortality decreased from 15.9 to 7.12 and 11.1 to 4.47 per 1,000 live births respectively between 2001 and 2009. However, given that at the time the majority of civil works were still ongoing, attribution for these outcomes remains ambiguous.
The Task Team also subsequently added that the ratio of inpatient care to ambulatory consultations decreased from about 36 in 2003 to 28 in 2009, showing an increased usage of cheaper ambulatory services.
To improve the health status of underserved populations elsewhere, with special emphasis on the poverty-stricken Northern region: Modest
An institutional strengthening and decentralization strategy was adopted and implemented in all 5 regions, meeting the target.
A Financial Management Unit was established within the MOH Planning Unit for contracting and managing performance agreements, meeting the target.
The project financed the establishment of: (1) a Health Information Unit in the MOH, which monitored basic health services; and (2) a unit for M&E of 12 priority programs, exceeding the target of establishing a Monitoring and Evaluation (M&E) Unit with the capacity to monitor basic health services and at least two priority programs.
The Human Resource (HR) Information System was strengthened through: (a) workshops and study tours on preventive/curative practices; (b) computers and software for MOH central and local levels and hospitals to improve personnel databases and recruitment; (c) support for regulations on MOH internal control of personnel; (d) a manual to train facilitators; and (e) non-economic incentives regarding mental/occupational health workers. This fell short of the target of a strategy for human resource management being designed, approved and implemented.
A new health code and proposal to organize the national health system were prepared and a national health law and regulations were approved, exceeding the target of a reformulated health code prepared.
The MOH launched and implemented strategies to improve care in obstetrics, gynecology and pediatrics in 30 hospitals, exceeding the target of 5.
The project promoted new waste disposal regulations, and helped get a national policy approved and implemented across the public health system (in all public hospitals and 102 primary health clinics), exceeding the target of 7 hospitals and 7 primary health clinics.
Government financing for basic health services in the Northern Region increased. By 2010 the MOH had absorbed the cost of 2,266 more staff and of service delivery into its regular budget. For this purpose, the Ministry of Finance allocated nearly US$ 19 million of additional resources.
284,000 people in the Northern Region, and 352,000 in earthquake-affected areas, were reached with basic health and nutrition services, exceeding the target of 150,000 and 200,000, respectively.
In 2008, NGO teams facilitated an average of 7 visits a month per community and 2.6 per capita contacts a year. . MOH teams made an average of 2 visits per community and 1.6 per capita visits in 2008, the latter of which rose to 3.5 by 2010. Since no baseline data were made available (original or Red Solidaria), no information on progress can be inferred. The target was a 50% increase.
6,000 primary care providers (MOH and NGO) were trained and equipped in delivery of basic health and nutrition services.,NGO providers received limited training. The target of fully trained and equipped government and non-government providers was partially met.
8 contracts were signed with 5 NGOs to provide basic services to cover at least 150,000 people, meeting the target of contracts signed to cover 150,000 people.
17 performance agreements reaching 352,000 beneficiaries were signed to implement outreach programs for the provision of basic health and nutrition services, exceeding the target of 6 performance agreements signed.
A Dengue surveillance system was established and the program implemented, meeting the target. A communication strategy on the prevention and control of HIV/AIDS, STIs and Dengue were developed and implemented via radio and TV campaigns, meeting the target.
Since 2005 all pregnant women were tested for HIV and all HIV-infected pregnant women received free treatment, exceeding the target of 25% identified as infected. All women seeking STI treatment were offered HIV tests, which exceeds the target of 50%. This was done with the assistance of the Global Fund To Fight Aids, Tuberculosis and Malaria (GFATM).
A substandard baseline survey was discarded, meaning that no baseline data are available for health sector indicators. Progress can, however, be measured using indicators from the Red Solidaria Program in 2007 (Red Solidaria, launched in March 2005 by the government, was a program to provide assistance to extremely poor families, implemented in the same poor municipalities that were part of the project). However, these data come with the caveat of a two-year time lag, and indicators are not the same as the ones originally envisaged.
Community awareness of Aedes Aegypti mosquito breeding control mechanisms increased from 72.7% in 2007 to 93.3% in 2010, exceeding the target of a 25% increase (91%).
DTP (diphtheria, tetanus, and pertussis) vaccination coverage rates have increased from 73.6% to 86.3% since 2007. The original target was a 20% increase from 2005 in the target population.
Pregnant women who receive five prenatal visits, iron supplements and tetanus vaccinations increased from 76.6% in 2007 to 79.2% in 2010. The original target was a 20% increase of at least two prenatal checkups from 2005.
The number of children attending five growth promotion sessions by age 1 increased from 86.3% in 2007 to 93% in 2010, exceeding the target of 90%.
Given a 10-year horizon, the PAD estimated the Internal Rate of Return to be 33.5%, the Net Present Value US$ 123 million, and the Benefit-Cost Ratio 1.76. Benefit estimates were based on: (a) reduction of average length of stay and improved operating efficiency; (b) cost savings through improved general health conditions; and (c) productivity improvements arising from a reduction in mortality and morbidity rates. Sensitivity risk analysis estimated that a 3-year delay, or 20% reduction in benefits, would bring the Internal Rate of Return down to 17%.
The ICR does not include a comparable computation at project completion. It provides no evidence of a reduction in length of stay at hospitals, improved operating efficiency, cost savings through improved general health conditions, or productivity improvements arising from a reduction in mortality or morbidity rates.
Efficiency gains were also expected due to economies of scale by adding the emergency component to an existing project. However, civil works cost 37% more than budgeted for the six hospitals due to an underestimation of prices, new seismic norms requiring different and more expensive construction materials, and expensive ex post modifications of hospital designs. Furthermore, the 4-year, 2-month project delay contributed to increased project management costs.
A costly works supervision firm (ESEO) performed very poorly and allowed the operation of non-performing firms.
Five out of six contracts were awarded to joint ventures with no history of working together, resulting in inefficiencies. The MOH faced four lawsuits from construction firms, costing US$ 9.73 million in arbitrations and settlements. One is still pending, which is expected to amount to a further US$ 2.4 million. The MOH managed to recover US$ 2 million in legal action against poorly performing firms.
Efficiency is rated modest.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated