Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Ht: Emergency School Reconstruction Project


  
1. Project Data:   
ICR Review Date Posted:
04/03/2013   
Country:
Haiti
PROJ ID:
P115261
Appraisal
Actual
Project Name:
Ht: Emergency School Reconstruction Project
Project Costs(US $M)
 5.0  5.1
L/C Number:
CH460, CQ666
Loan/Credit (US $M)
 5.0  5.1
Sector Board:
Education
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  03/05/2009
 
 
Closing Date
12/30/2011 05/31/2012
Sector(s):
Primary education (90%), Other social services (5%), Public administration- Education (5%)
Theme(s):
Education for all (65% - P) Natural disaster management (35%)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Susan Ann Caceres
Judyth L. Twigg Ismail Arslan IEGPS2

2. Project Objectives and Components:

a. Objectives:


    According to the project paper (p. 6) and the Financing Agreement (p.5), the development objective was "to assist the Government of Haiti in restoring and improving access to basic education in selected destroyed and/or heavily damaged public primary schools of its territory."

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

The project contained two components:

1. Building back better selected destroyed and/or heavily damaged public primary schools (appraisal, US$ 4.3 million equivalent; actual, US$ 3.8 million equivalent) was to intervene in schools that had been destroyed and/or schools whose infrastructure represented a serious risk for the safety of students and teachers. It was to upgrade facilities in a few selected schools that were used as temporary shelters in case of natural disasters. There was to be a process to consult/inform communities of the reconstruction. The project was estimated to finance the reconstruction/rehabilitation of 15 primary schools out of an estimated 964 schools partially or completely destroyed by the hurricanes identified by a national damage and needs assessment. The project was to use better standards and building norms, and designs were to include access to latrines and potable water. Schools were to be selected based on specific criteria (schools with at least 50 percent of their classrooms destroyed and/or schools in serious risk of collapsing) and geographical targeting. The project was also to upgrade 5 schools to be used as temporary emergency shelters by building an extra-large room attached to each existing school structure; upgrading and expanding the school latrines; and ensuring that schools had secure access to drinkable water and /or an energy source.

2. Reducing and mitigating the vulnerability of school infrastructure (appraisal, US$ 0.7 million equivalent; actual, US$ 0.4 million equivalent) was to support the creation of a mechanism for increased preparedness of school facilities for natural disasters. This included: (1) National Action Plan for Safe Schools; (2) capacity building of the Civil Works Unit; and (3) development of a communication strategy and communication tool to disseminate the main lessons learned. Capacity building of the Civil Works Unit was to be done so that staff would properly enforce new norms and regulations relating to the construction and maintenance of educational infrastructure. Furthermore, "to measure the demonstration effect of the Emergency School Reconstruction Program (ESRP), the utilization of the new methods for safe school construction by education stakeholders and donors" was "to be tracked during the implementation of the Project" (PAD, pp. 9-10).

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost: In addition to the above noted costs for each component, project management [Fund for Social and Economic Assistance (FAES)] amounted to US$ 0.5 million, and the Project Preparation Facility was US$ 0.40 million. The Project Preparation Facility was requested by the Government to reimburse expenditures incurred to equip the Civil Works Unit, to prepare the list of schools to be reconstructed and for the FAES to prepare and evaluate the prequalification dossiers of proposed sub-projects, and to finance a project assistant for the FAES (PAD, p. 18). Actual costs were slightly higher for component 1 (US$ 0.18 million) than estimated, and the total number of rebuilt schools decreased as a result of increased demand for construction after the January 2010 earthquake (ICR, p. v).

Financing: The project was financed by an IDA Grant.

Borrower Contribution: The Borrower made no contributions, nor were any planned.

Dates: The project was extended five months from its original closing date to May 31, 2012 to allow for completion of school construction that had been delayed due to complications at construction sites during the rainy season in 2011.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

High. In 2008 Haiti was struck by four successive storms and hurricanes that severely damaged public infrastructure and injured and killed hundreds of people. For the education sector, the estimated damages and losses were nearly US$ 30 million, with another US$ 70 million in identified needs (ICR, p. 1). 964 schools were greatly damaged, impacting more than 200,000 children. These estimates comprised the public schools, which accounts for only 20% of the total. There had also been a period of neglect of school buildings, resulting in a large deficit in school infrastructure. In September 2008, the Government passed the State of Emergency Law, which allocated US$ 4 million to emergency relief for schools (ICR, p. 27). There were already two previous education projects implemented by the Bank that addressed both demand-side intervention in the form of tuition subsidies and school nutrition (Education for All Adaptable Program Grant, 2007-2012, US$24.8 million), and supply-side interventions through newly trained teachers (Meeting Teacher Needs, 2008-2012, US$5.83 million). Neither of these projects addressed school reconstruction, which was a pressing need after the storms.

The project's objective was consistent with the Bank's current Interim Strategy Note (2012-2013), which emphasized "quick wins" through the project's objective of restoring access to schooling through the reconstruction of schools. The objective remains highly relevant as Haiti continues to rebuild schools that were damaged in hurricanes and the January 2010 earthquake.

b. Relevance of Design:

Modest. Project design addressed the immediate need for access by rebuilding schools and also included activities to mitigate future natural disaster vulnerabilities through the development of standards for safe school construction. Activities related to the development of safe school standards, such as training to improve the capacity of the supervisory agency, the Ministry's Civil Works Unit, would be needed to accomplish the project's aim. However, given the short duration of the project, the other planned activities intended to increase preparedness within the education sector for future disasters, including the development and adoption of a National Action Plan for Safe School Construction, were overly ambitious beyond the scope of the project's focus and objective. With the exception of the activities to mitigate vulnerabilities of the education infrastructure, the other activities were logically aligned with the specified development objective and outcomes, but the Results Framework was weak.


4. Achievement of Objectives (Efficacy) :

Restoring and improving access to basic education in selected destroyed and/or heavily damaged public primary schools: Substantial

Outputs:

11 schools were rebuilt or rehabilitated, which met the downwardly revised target (11). The numerical target was revised downward due to cost increases and delays related to the January 2010 earthquake. The ICR reports that schools were built within anti-cyclonic and anti-seismic norms. An independent assessment of the construction, however, found a number of minor and major issues with the design and quality of construction. One concern was the fact that "the metal structures and roof on the rebuilt schools were not specially reinforced to withstand a hurricane" (ICR, p. 48). (See Section 5 for discussion of construction quality.)

According to the project team, the project rebuilt/renovated 71 classrooms out of 81 targeted classrooms in 11 schools. 10 classrooms did not need to be rebuilt because they were not damaged, and therefore the project renovated all the targeted classrooms and restored access.

4 schools were retrofitted to have emergency temporary shelters, meeting the downwardly revised target (4). The ICR indicates that the target was revised downward due to factors related to the 2010 earthquake. The ICR (p. 9) indicates that these shelters were used in September and October 2012 by an unspecified number of families.

100% of rebuilt schools had prequalification visits, meeting the target (100%). One quality assurance supervision was carried out by the Civil Works Unit for each school built, which did not meet the original target (3 per school) or revised target (2 per school). This finding is based on staff reports, as no documentation after each visit was submitted to the implementing agency, the Fund for Social and Economic Assistance (FAES), despite requests (ICR, p. 17). The project team also reports that staff from FAES's regional offices were at each school on a weekly basis monitoring construction progress and quality.

The ICR also reports on activities related to disaster preparedness and resilience:

20 consultations and 5 workshops were held to increase stakeholder awareness of vulnerabilities to disaster and mitigation measures for the education sector. The ICR reports that tools developed by the project were used to conduct vulnerability assessments in 158 schools in two departments, and shared at national workshops and other smaller workshops. Feedback from these workshops validated findings and informed the development of the draft National Action Plan for Safe Schools (NAPSS).

A draft NAPSS was prepared in 2011 and is being revised, which partially met the target. The plan has been presented to donors but has not been adopted by the Government. The elements of the plan are: (1) prioritization of needs for new school buildings, based on international standards and on the findings of the vulnerability assessment and study; (2) addressing the issue of retrofitting of existing school buildings; (3) protection and safety of nonstructural components of schools, such as equipment and pedagogical materials; (4) national guidelines for emergency planning at the school level; (5) guidelines for capacity building of education professionals to promote a culture of safety in schools; and (6) clear and succinct guidelines to ensure effective implementation. The ICR (p. 17) notes some weakness in the document: it does not include a political framework for major decisions, and the method for assessing school vulnerability is not based on objective criteria. The NAPSS was presented at a workshop attended by 100 people (ICR p. 34). Given that the NAPSS has not been adopted, the project did not track the use of new methods for safe school construction by education stakeholders and donors (PAD, p. 9-10).

24 Civil Works Unit staff were trained to become Master Trainers on NAPSS issues, exceeding the original (20) and revised (10) target. This training was done by staff within FAES with content derived from seven training modules developed by the project. The modules covered evaluating vulnerability and risk, anti-hurricane and anti-seismic norms, team work, project management, social accountability, the role of school facilities, and parent-teacher relationships.


Outcomes:

Rebuilt schools maintained an 85% occupation rate within 6 months of project closing, exceeding the target of 75%. The Government reports that these schools can serve 3,550 primary students based on the 71 renovated classrooms (ICR, p. 33). The schools are located in poor rural areas. The project team shared updated enrollment data for the 2013 school year, showing that the rebuilt schools attained an occupancy rate of 92% from the pre-construction enrollment, exceeding the target of 75%.

The project team reported that actual enrollment for the current school year was 2,994 students, not using the full capacity for 3,550 children. While the project exceeded the target occupancy rate, the project team provided two explanations for why the schools were not fully enrolled. First, the demand study was done before the earthquake occurred and people were displaced, and it is also difficult to estimate demand because of a high level of dropping out among over-age students. Second, this school year the Government instituted a new program to pay school fees at private schools, which may have shifted some of the enrollment to private schools.


5. Efficiency:

Efficiency: Modest
No economic analysis is presented in the ICR.

An earthquake occurred on January 10, 2010, approximately seven months after the project became effective, creating more infrastructure destruction and halting implementation for several months. After the earthquake, construction costs increased because of increased demand for construction material and labor, and because of subsequent geotechnical surveys and anti-seismic measures that were added (ICR, p. 33). The unit cost of school construction increased from US$200,000-US$250,000 to US$ 300,000-US$ 350,000, which was 40-50% higher than estimated. The ICR does not provide a comparison of costs of similar construction from other donors during this time period. In addition to reducing the number of schools built, design features such as fences, playgrounds, outdoor facilities, and improved toilets were eliminated, but some of these features were later reinstated in some schools, financed by gains from exchange rate fluctuation and resources from the Fund for Social and Economic Assistance (ICR, p. 5 and p. 32).

The ICR reports two sets of information on the quality and resilience of school construction. At one point in the text (p. 10), the ICR reports that schools were constructed using anti-seismic and anti-hurricane standards, which increases the longevity of the schools and reduces the likelihood of interruption of school attendance from natural disasters. The ICR reports that these schools sustained minor or no damage during recent hurricanes in September and October 2012. The ICR (p. 10) suggests that more resistant buildings will lower maintenance costs.

However, the ICR also reports on an independent architect's review of construction quality that found that "there were a number of minor and major flaws in the design and the construction of the schools," such as: (i) the cables that secure school roofs were not properly positioned, (ii) iron was used in place of steel, and (iii) there were inferior water runoff channels (ICR, p. 6). There were quality concerns with the ceilings, toilets, sinks, kitchens, and other areas (ICR, pp. 43-48). Moreover, "metal structure and roofs were not specifically reinforced to withstand a hurricane, and there is no drainage under the eaves and the gutters are not solidly attached...making it likely the schools would suffer heavy damage and flooding" (ICR, p. 48). If this assessment is correct, there will be increased costs in the future to rectify the shortcomings as well as additional maintenance costs.

In the region's comments to the draft ICR review, they reported that the Fund for Social and Economic Assistance fully addressed the weaknesses identified by the independent architect by using resources from the Fund for Social and Economic Assistance or those of the construction firms performing the work.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of objectives was high, but relevance of design was modest. There was substantial attainment of the only objective, and efficiency was rated modest. Project design contained activities on the development of national safe school standards that were not relevant to the achievement of the project's development objective. The project constructed 11 primary schools and 5 shelters, resulting in 71 reconstructed classrooms which, in combination with 10 undamaged classrooms, restored full access for the 81 targeted classrooms. The enrollment rates in these classrooms exceed the targeted enrollment rate of 75%.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

Other donors, such as the Inter American Development Bank, are continuing to finance school construction. While the project helped to assess the disaster resilience of school buildings and increase awareness of the importance of school building safety, there are a number of risks to the development outcome. The project was not able to institutionalize the National Action Plan for Safe Schools, despite recognition that the cause of two school tragedies in 2008 (schools collapsing and killing students and teachers) was "shoddy craftsmanship in the construction industry and absence of building codes and regulations, and the public authorities' inability to oversee and enforce school construction standards" (ICR, p. 26). As a result, there is lack of coordination across donors and lack of consensus within the Government to implement consistent, safe building standards and to adopt a strategy for the assessment/certification of school infrastructure. There are also risks to maintaining the schools built under this project, since there is no budget provision for maintenance. Instead, communities are expected to "monitor the conditions" of the schools without the necessary resources. Available funding to operate a school is low, and these resources are now expected to be prioritized for maintenance. There also continue to be significant capacity and leadership issues. The Ministry of Education did not provide the necessary leadership to adopt the National Plan for Safe Schools. Capacity building is still needed for the Fund for Social and Economic Assistance and the Civil Works Unit of the Ministry of Education, since schools constructed under this project had numerous minor and major design/construction issues. All of these risks combined pose a very high risk to the development outcome.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:

The Bank responded rapidly to the pressing need to rebuild schools. The Bank learned lessons from previous emergency response projects in other countries, including the need to ensure community engagement in design and implementation, and the importance of not requiring communities to provide in-kind or cash support for the rebuilding. Building norms for schools in Madagascar were used for this project, including: (i) reconstructed schools should be more resilient to natural disasters; and (ii) the new schools should have access to latrines and potable water. The Bank emphasized creating a maintenance budget for the new schools and suggested the use of debt relief resources as a source (PAD, Annex 9); however, nothing in this area was established. The Bank worked with various stakeholders to develop a balance between affordability and quality. The Bank committed to providing more supervision missions (4-5 per year) than is typical (2 per year), given the nature of the project. The Fund for Social and Economic Assistance and Civil Works Unit were involved in preparation.

Given the "lack of up to date construction norms and standards which guide the design and construction process, as well as institutional capacity for monitoring and adherence to the standards" (PAD, p. 57), it could have been foreseen that Fund for Social and Economic Assistance (in charge of civil works) and the Civil Works Unit of the Ministry of Education (general quality control) would need more support to ensure that schools were built to meet standards; however, according to the project team, it is not clear that any capacity beyond these two entities existed. Designs were established during appraisal without consideration of seismic norms. While the Bank participated in some of the initial presentations of school design, it did not verify the technical design, which could have minimized subsequent issues (according to the project team). There were inadequacies in the analysis of the political economy environment and in the process established to adopt the National Plan for Action for School Safety. The risks related to implementation capacity and sustainability were rated substantial and the technical approach of "building back better" was rated high, but the mitigating factors were inadequate (PAD, page 23).

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:

The Bank ensured stakeholder participation in the task force that developed the National Action Plan for Safe Schools, which has been presented to donors, and is still under revision. However, the National Action Plan for Safe Schools was to be overseen by a task force that did "not get off the ground" (ICR, p. 34) and so was replaced with an ad hoc committee. As a result, there was no political mechanism to ensure ownership and adoption of the Plan, and it has not been finalized or adopted. Since the Ministry of Education was in the lead on this process, it is not clear what role the Bank could or should have played.

The project team reported that the increased number of supervisory missions promised in preparation were conducted by the Bank team. There were three Task Team Leaders (TTLs) during the course of the project. The Government reports that the transitions between TTLs were not smooth, creating delays and misunderstandings (ICR, p. 35); however, the project team did not concur, noting that the Government was notified of changes, and handover missions occurred. The project team indicated that each TTL had a different style and different preferences in the processing of no objection requests, which created challenges for the Implementing Agency but did not negatively impact implementation.

The Bank team correctly restructured the project to revise targets downward due to rising costs. However, the Bank should have been more proactive in establishing a Government or donor financed maintenance budget for the schools; instead, there was no provision and no plan at project closing. It is also not clear why the Bank waited until the end of the project to bring in an independent architect for an assessment.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government allocated approximately US$4.5 million for emergency school reconstruction and rehabilitation activities identified by the Ministry during the damage and needs assessment. An agreement was signed between the Economic and Social Assistance Fund and the Ministry of Education and Vocational Training to carry out activities in the School Reconstruction Program. This agreement delineated their respective role/responsibilities (ICR, p. 27). During project preparation, the Government provided funds (US$ 400,000) to acquire equipment for the Civil Works Unit of the Ministry to cover the costs of identifying the schools to be rebuilt and of preparing and assessing pre-qualification (ICR p. 28). The Government was later reimbursed for these funds by the project. It is commendable, given the level of devastation after the earthquake, that attention was not diverted to other priorities; the Government and the implementing agencies remained focused on the project. The Government was supportive and helpful and maintained a high level of commitment to the project and basic education. The ICR reports that there were no issues with the Ministry of Finance. The President inaugurated several of the newly built schools. However, according to the project team, when the Minister changed, there was weak ownership in the Ministry of Education for the National Action Plan for Safe Schools. This document still has not been finalized or adopted for future building projects.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

The Fund for Social and Economic Assistance (FAES) was the implementing agency. It had all fiduciary responsibility for the project and for managing the construction firms. The executing agency, the Ministry of Education, also had specific roles within the project. The Civil Works Unit (DGS) within the Ministry of Education monitored the quality of construction, and the Ministry itself had responsibility for the National Action Plan for Safe Schools.

The offices of one of the Civil Works Units was badly damaged by the earthquake and had to be temporarily relocated to borrowed space for several months. The Fund for Social and Economic Assistance did not suffer large damage, and so its capacity was not affected. Despite the overall devastation and disruption caused by the earthquake, school construction was completed with only a modest delay and in time for the 2012/2013 school year.

The Civil Works Unit worked with the Department for Private Education and Partnership to select eligible schools based on specific criteria and geographic targeting as well as beneficiary consultation. During this process, the Civil Works Unit collected information related to the number of children that had been regularly attending the schools that would be rebuilt to assess potential unmet demand. The prequalification of schools was done well. There was only one visit to each school from the Civil Works Unit to monitor construction. FAES's regional offices provided weekly inspections of each school. An independent architect's review of construction quality found that "there were a number of minor and major flaws in the design and the construction of the schools" (ICR, p. 6), pointing to important shortcomings in the performance of both FAES and the Civil Works Unit of the Ministry of Education. In the region's comments to the draft ICR review, they reported that the Fund for Social and Economic Assistance fully addressed the weaknesses identified by the independent architect by using resources from the Fund for Social and Economic Assistance or those of the construction firms performing the work.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

Data were used to determine the scope of school construction. A school enrollment figure was estimated, and the project's monitoring and evaluation framework developed an indicator to later measure the occupation rate in comparison to what was expected, which was then summed and averaged for all the schools. However, this analysis does not show the actual or increased enrollment and does not provide information per school.

b. M&E Implementation:

Construction progress was monitored from the Fund for Social and Economic Assistance field offices to the central office.

a. M&E Utilization:

Data on the cost of construction were used to revise the number of schools constructed.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:

The Project Paper (p. 56) reports that the prescreening of the project triggered Environmental Assessment (OP 4.01) and Involuntary Resettlement (OP 4.12). A Resettlement Policy Framework was to be prepared within four months of the effectiveness date of needing the land. The ICR (p. 7) reports that no involuntary resettlement occurred, since construction was done on existing school sites. The project was classified category B for Environmental Assessment. An Environmental Management Framework was developed and disclosed. The project team reported that there was a staff member in the Environmental Unit who was responsible for monitoring the application of the Environmental Framework. The Borrower's comments (ICR, p. 32) report that monitoring visits focused on waste management, water management, worker protection, dust emission, civil protection, and worker education. Staff from the Environment Unit identified actions needed to address any environmental problems encountered (ICR, p. 32) and complied with environmental safeguards.

b. Fiduciary Compliance:

The ICR reports no issues with compliance with the Bank's financial management procedures. FAES was in charge of financial and procurement management for the project. The procurement plan was approved by the Bank. The ICR states there were no issues or delays with procurement and financial management procedures. Audits were not qualified and at the end of the project there were no outstanding reports.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:
None



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
High
High
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

IEG's interpretation and synthesis of the main lessons presented in the ICR (p. 14) are:
  • Even though a country may be motivated to improve safety, efforts to adopt a safety strategy and standards may lag. It proved difficult to develop a comprehensive Plan for Safer Schools with this project. This failure related to lack of commitment from the Ministry to enact changes. As a result, the Ministry did not exhibit strong leadership, and so the demonstration effects from the project for subsequent school building were blunted.
    • In projects that build schools, the Bank must establish an early review process of the designs to ensure adherence to standards. This implies that the Bank must invest resources into this activity and, by doing so, improve on the local design. It also means that adequate supervisory resources by the Bank, including an architect/engineer, need to be employed to ensure that designs are followed.

  • 14. Assessment Recommended?

    No

    15. Comments on Quality of ICR:

    The ICR provides a candid assessment of implementation weaknesses, particularly related to the Bank and Borrower/Implementing Agency. Annex 10 (Excerpt from the Independent Architect Report) is candid, and the ICR team should be positively acknowledged for its inclusion. The ICR describes implementation thoroughly and provides an evidence-based assessment of attainment of the project's development objective. The ICR does not state how the Bank monitored compliance with the Environmental Framework, subsequently clarified with the project team. The comments submitted by the Borrower were very helpful and supplemented information presented in the ICR.


    a. Quality of ICR Rating: Satisfactory

    (ICRR-Rev6INV-Jun-2011)
    © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions