Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Gaza Emergency Water Project


  
1. Project Data:   
ICR Review Date Posted:
09/30/2013   
Country:
West Bank & Gaza
PROJ ID:
P065920
Appraisal
Actual
Project Name:
Gaza Emergency Water Project
Project Costs(US $M)
 22.90  30.41
L/C Number:
Loan/Credit (US $M)
 20.00  30.41
Sector Board:
Water
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  06/07/2005
 
 
Closing Date
01/31/2009 01/31/2012
Sector(s):
Sub-national government administration (67%), Water supply (15%), Sanitation (12%), Sewerage (6%)
Theme(s):
Pollution management and environmental health (29% - P) Other urban development (29% - P) Rural services and infrastructure (14% - S) Other human development (14% - S) Conflict prevention and post-conflict reconstruction (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Toneema M. Haq
Roy Gilbert Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The Trust Fund Grant Agreement (Schedule 2) states the project objectives: "(i) to improve the efficiency, management; operation and delivery of water and wastewater services in Gaza; (ii) to build capacity of CMWU [Coastal Municipalities Water Utility]; and (iii) to strengthen the regulatory and institutional capacity of the PWA [Palestinian Water Authority] ."

The Memorandum of the President (MOP) states: "The primary objectives of the project were to improve the provision of adequate water and wastewater services in Gaza, and to functionally establish the Coastal Municipalities Water Utility as an autonomous well-run modern utility. In addition, the project has a secondary objective to strengthen the PWA's capacity to assume sector regulatory functions" (p. 6).

This evaluation will use the objectives as stated in the Grant Agreement and retained in three subsequent Project Papers for additional grant funding. It is more specific and monitorable.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project had three components.

Component 1: Performance-based Management Contract (Estimated Cost: US$8.6 million; Actual Cost: US$4.5 million)

This component financed the fees for the private sector contract as well as performance incentive fees based on meeting agreed targets. These targets included establishing the new utility for water and wastewater, improving water quality by upgrading the existing disinfection program, increasing water quantity by reducing water losses and increasing supply capacity, and improving the management system of water and wastewater services. During the 2008 restructuring, the performance-based management contract was terminated and the remaining funds (US$1.495 million) were reallocated to cover civil works and goods for Component 2.

Component 2: Capital and Operating Investment Fund (Estimated Cost: US$12.0 million; Actual Cost: US$22.5 million)

This financed materials, supplies and equipment for the operations and maintenance of water and wastewater systems. Specifically, the Fund financed drilling of water productions wells, supply of chemicals, replacement of meters and main pipes, installation of meters (domestic and public), maintenance of sewer lines, rehabilitation of wastewater treatment facilities, repair of equipment for water desalination plants, construction of new administrative facilities, and incremental costs of water and wastewater facilities. This Fund was under the control of the private sector operator and received reallocated funds from Component 1.

Component 3: Institutional Reform and Capacity Building (Estimated Cost: US$2.3 million; Actual Cost: US$4.0 million)

This financed the establishment of the Coastal Municipality Water Utility (CMWU) and the Project Management Unit. Funding was also provided to the Palestinian Water Authority (PWA) to put in place an effective regulatory framework. These were not recurrent costs.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost. The total project cost increased from US$22.9 million at appraisal to US$31.0 million at completion. The Bank provided additional grant financing of a total of US$11 million. This was mainly in response to the border closing and a severe reduction in the availability of goods and materials resulting from the unrest and separation of Gaza from the West Bank following the election of Hamas in 2007. The first additional financing of US$5 million was approved in June 2008 and financed operations and maintenance costs of water and wastewater facilities for an additional year beyond the original closing date and cost overruns of the CMWU. The second additional financing of US$3 million was approved in July 2009 and financed CMWU operation costs following damage to facilities due to conflict in December 2008. The third additional financing of US$3 million was approved in April 2011 and financed CMWU's targets and operational expenses for an additional year.

Financing. The project is financed by the Trust Fund for Gaza and the West Bank established in 1993 to finance Bank assistance there. There were no co-financers for this project, but other donors supported the utility in parallel to Bank financing. There were 22 parallel financing agencies, including the European Union, Oxfam, and the United Nations Development Program, but the ICR does not give details on what they financed and how much. The Project Team later clarified that the CMWU managed US$150 million of parallel donor funds.

Borrower Contribution. At appraisal, the borrower was committed to financing US$2.9 million (13% of total costs) but at completion the borrower financed none of the project cost. According to the project team, this was due to the termination of Palestinian Authority involvement after the separation of Gaza from the West Bank in 2007.

Dates. The original closing date was 01/31/2009 but the Project actually closed three years later on 01/31/2012. The project restructuring in 2008 extended the closing date to 01/31/2010. The second additional financing of July 2009 had a closing date of 12/31/2010 which was extended to 06/30/2011. The third additional financing of April 2011 closed on 01/31/2012 which was the actual completion of the project.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
The Interim Strategy Note (ISN) for the West Bank and Gaza 2012-2014 highlights that the improvement of water and sanitation services falls under the Fourth Pillar (Support Public Infrastructure Development) of the Bank's strategy. This pillar is "the most important in terms of the operational portfolio" (ISN, p. 9). Rehabilitation of the "dilapidated" (p. 12) infrastructure of Gaza, which includes water, is a priority of the Extraordinary Replenishment for the Trust Fund approved in June 2011. Institutional strengthening in the water sector, one of the objectives of this project, is highlighted in the ISN. Investment in the water sector continues to be highlighted in the country strategy (Palestinian Reform and Development Plan 2008-2010).

Relevance of Objectives is High.

b. Relevance of Design:
There was a results framework in the Technical Annex to the MOP which laid out how the project objectives were to be achieved. Each objective was associated with specific outcomes which were relevant to its achievement.

The objective of improving the provision of adequate water and wastewater services in Gaza was associated with two outcomes: drinking water delivery meets quality and quantity standards and wastewater collection and disposal services meet standards. The second objective of establishing the CMWU as an independent modern utility was associated with setting clear objectives for the CMWU and monitoring performance of its services through action plans. Finally, strengthening the capacity of the PWA to assume its regulatory role is associated with the PWA regularly monitoring water abstraction, groundwater and wastewater qualities and publishing reports about compliance with standards.

Components 1 and 2 fit within the first project objective, and Component 3 fit within the second and third project objectives. The results framework annex included performance standards for the management contract which would govern the private operator. This further elaborated upon how to achieve project objectives.

Relevance of Design is Substantial.


4. Achievement of Objectives (Efficacy) :

Efficacy is assessed in three parts:

Objective 1: Improve the efficiency, management; operation and delivery of water and wastewater services in Gaza is Modest.

Outputs

1. The ICR reports that disinfection of drinking water in the network increased from a baseline in 2005 of 99% to the target value in 2011 of 99.8%. IEG does not see this as a significant change, and furthermore could not find any reference to a project justification for setting an improvement to more than 99% as a meaningful performance target for an operation of this kind, taking into account statistical measurement errors likely to overshadow such small changes. This is an indicator of limited value since it does not inform us if the concentration of chlorine was adequate for disinfection. The Project Team, however, maintained that achieving a level of chlorination above 99% is "an essential element of the utility performance to eliminate the possibility of water-borne disease."
2. By 2011, 95% of wastewater collection and disposal service met standards. This was more than the 85% target value set but less than the 100% original target value set in 2008. However, it is hard to judge improvement since the baseline value in 2005 was "not determined" (ICR, p. iv). The Project Team maintained that by the end of the project the new and rehabilitated wastewater facilities collected and treated "almost the entire volume of waste water and [thus] eliminating random disposal of sewage."
3. The percent of time that water wells are operational went from a baseline of 88% on 2005 to 90% in 2011, achieving the original and revised target values. This is another very small change that may not have been worth pursuing and which may be clouded by possible data measurement errors. The Project Team maintained that there is little risk for data measurement errors in this indicator since the operating hours of the water wells were determined based on the real time the wells were in operation.
4. The volume of water billed to volume of water produced did not change from its baseline of 62% in 2005. The original target was 76% and the revised target was 70% but the percentage remained at 62% as of 2010 when it was last measured. The project therefore failed altogether to deliver the intended improvement. The reason for this was later explained by the Project Team. While the project reduced technical losses (leakage in the networks, inaccurate meter readings, etc.), despite installing meters on network segments to locate illegal losses (from theft), absence of law enforcement made this impossible to achieve.
5. The number of hours all customers receive water each day changed little from the 2005 baseline value of 6 hours, largely because of frequent power cuts, according to the ICR. This is another case of the project failing to deliver an intended output. However, the Project Team maintained that the population in Gaza copes with this by having sufficient installed storage capacity on the roofs of homes and businesses.
6. Wastewater effluent quality standards went from an undetermined baseline in 2005 to 95% in 2011, surpassing the revised target of 85% but less than the original target of 100%. The ICR did not elaborate on what this indicator meant and what the standards were. The Project Team provided some information on the biochemical oxygen demand (BOD) levels: "the existing facilities have been removing a total load of BOD that is multiple times of what was originally specified in the sewage quality performance formula." However, without baseline, target and achievement data on BOD, it is difficult to assess what the project achieved for effluents quality.
7. 50 water production wells were drilled or rehabilitated.
8. Over 30,000 meters of old asbestos main pipes were replaced.
9. 15,000 residential water meters and 20 public meters were installed.

Outcomes

1. Wastewater collection and disposal services met standards, however there was no baseline value and no description of standards in the ICR. The performance standards were spelled out in the technical annex to the MOP (from 2005) but it is unclear if these standards remained constant throughout the project until its closing in 2012.
2. Wastewater effluent quality met required standards, but this outcome is subject to the same caveats as outcome 1, above.

Though most of the outputs under this objective did not improve appreciably (taking into account statistical errors), or cannot be determined to have improved (since there was no baseline provided), the objective was somewhat achieved (water wells drilled or rehabilitated, pipes replaced, meter installed) taking into account the difficult political situation in Gaza.

Objective 2: Building capacity of CMWU is rated Modest.

Outputs

1. In 2005 CMWU did not exist and during the course of this project it:
a. Rented and furnished regional offices.
b. Set up technical and administrative departments.
c. Set up a payroll system.
d. Finalized water and wastewater billing systems.
e. Implemented a customer services information system.
2. Annual gross revenue collection increased from a baseline of 40% in 2005 to 35-55% in 2011. The target was set at 50%. The uncertainty in the final value makes it difficult to determine if this target was achieved, and it is possible that the actual value at completion may have decreased. Collection rates remain low due to poverty and unemployment.

Outcomes

1. From not existing physically at project appraisal, CMWU is an autonomous and functioning utility
2. The utility sets objectives and monitors its services through action plans on investment needs and priorities
3. CMWU took a leading role in dealing with emergencies to mitigate flooding risks associated with the accumulation of wastewater in Beit Lahia, north Gaza (ICR, p. 15).

However, CMWU has been unable to build up capacity to integrate with municipalities due to the political situation. Only one municipal utility is fully integrated into the CMWU, thirteen are partially integrated and eleven are independent. Integration of utilities is necessary for improvements in revenue collection. The Project Team later clarified that CMWU is operating water and wastewater services "in a large number of municipalities."
Due to security concerns, the private operator could not manage the project from within Gaza itself (ICR, p. 9).


Objective 3: Strengthen the regulatory and institutional capacity of PWA is Negligible.

Outcome

1. The PWA improved its sector regulatory role supervising CMWU, communications, and wells licensing arrangements during the early stages of the project. However, following the separation of Gaza, work on and funding of this component ceased and the project could no longer achieve this intended objective.

5. Efficiency:

There is no Economic Rate of Return (ERR) calculated at appraisal as it was prepared as an emergency project. An ERR could potentially have been calculated at project completion, but due to the unstable country situation after 2008 in the Gaza, it was probably impossible to collect the data needed to estimate it. A financial rate of return could have been attempted at project completion but was not.


There is a qualitative review in the ICR which "concludes that the project was cost effective, given the unique operating contest." (p. 24), but gives no evidence of the actual costs. There is no data available for cost-benefit analysis due to the political and economic crises in Gaza after its separation. The financial position of the utility was undermined due to collection inefficiency, unemployment and poverty, and limited area of revenue collection.

The project was delayed by three years due to the additional grant financing. By the time of project closing, the original emergency situation (the Palestinian Intifada of 2000-2005) may not have been relevant to the project but there was another emergency situation during the project (separation of Gaza from the West Bank in 2007) which was still relevant. The delays led to inefficiency but were understandable given the context.

Efficiency is Modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Based on High Relevance of Objectives, Substantial Relevance of Design, Modest Achievement of two objectives, and Negligible Achievement of one, and Modest Efficiency the Overall Outcome Rating is Unsatisfactory. This is mainly due to the non-achievement of one of the three project objectives.

a. Outcome Rating: Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

This project has all the economic, political and financial risks one would expect in the crisis-ridden West Bank and Gaza. The separation of Gaza in 2008 only exacerbated these risks due to border closings and severe reduction in the availability of materials. High unemployment makes it difficult for people to pay existing user charges, let alone increased water and wastewater tariffs. Institutional reforms have been stalled by the unstable politics and uncertain security. These conditions have made sustaining a relationship with a private operator impossible and are likely to continue to do so. The financial viability of the implementing agency continues to be at risk since collection rates have not improved.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
The Bank prepared this project incorporating lessons learned from the previous project, the Water and Sanitation Services Project (FY1997), rated Satisfactory for overall outcome in the ICR and Moderately Satisfactory by IEG. From this earlier project the Bank ensured that the current project design was simple, that there was borrower commitment and an effective management contract in place. The prior engagement helped the Bank better understand risks and difficulties of a long-term private sector involvement in the water sector in Gaza. A short-term (three year) management contract was considered appropriate to create "an environment that had the proper conditions for private sector participation and risk taking" (ICR, p. 8).


There was no ERR calculated as this was an emergency operation. Safeguards were properly addressed (see Section 11a, below) and procurement and financial management were adequately included in project design. One main shortcoming of quality at entry was the lack of baseline data in the monitoring and evaluation framework for tracking indicators.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:
The Bank consistently ensured compliance with procurement, safeguards and fiduciary issues despite the disruption caused by the separation of Gaza in 2007 and subsequent border closing. When access into Gaza was tightly restricted, Bank procurement supervision was done through video conferencing and reviews of document provided by CMWU. It is unclear if on-site supervision ever resumed. The Bank office in Gaza "provided continuous support and engagement with the CMWU" (ICR, p. 19). Supervision of the project led to recognizing the need for three additional financing loans so that original objectives could be realized, despite the challenging country circumstances.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Palestinian Authority (PA) is considered to be the Government in this project. Up to the split with Gaza in 2008, the PA was committed to achieving the project's objectives and it enabled project implementation by coordinating financing management and procurement. Its performance pre-2008 is considered satisfactory, However, the political instability in Gaza led to opposition to project implementation "which significantly impaired the achievement of the project's objectives." (ICR, p. 19). Although the political issues were outside its control, the PA did not maintain an enabling environment for the development of an institutional framework. By project closing, the PWA was "not as effective as it had been during project preparation" (ICR, p. 15), primarily due to the separation of Gaza. Thus, after 2008 the PA's performance would be considered unsatisfactory.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
There were two implementing agencies during the course of the project. The PWA was the implementing agency from FY2005 to FY2008. After the separation of Gaza, the PWA could not perform and the CMWU became the implementing agency until project closing.


The PWA carried out it's functions adequately until CMWU took over after the separation of Gaza. The CMWU showed commitment and successfully attracted parallel financing from donors. However, the CMWU became "heavily dependent" (ICR, p. 20) on donor financing which was targeted to specific areas in response to emergency situations. Instead of CMWU becoming a regional utility with long-term goals, it became a utility that provides repairs and chlorination for the municipalities but is unable to integrate local utilities into it's organizational structure. Since it has not been able to raise tariffs due to high unemployment and the political instability, CMWU is not a "financially sustainable service provider." (ICR, p. 17). As much of the under performance of the utility is due to circumstances beyond its control (high unemployment, widespread poverty, blockade on Gaza, lack of law enforcement), the rating is assessed as moderately unsatisfactory.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
A results framework was prepared at appraisal in the Technical Annex of the MOP. Indicators were identified baseline values were not included. Six months after approval, baselines were included in the second Implementation Status Report.


There are three indicators measuring project objectives: (i) services consistently meeting standards (measured by drinking water delivery and waste water collection and disposal services meeting standards); (ii) CMWU fully functional and operational; (iii) capacity of PWA strengthened. In addition, there are five intermediate outcome indicators: percent of time when water wells are operational, volume of water billed to volume of water produced, number of hours all customers receive water daily, waste water effluent quality standards, and annual gross revenue collection. The results framework in the Technical Annex included performance standards for the management contract which would govern the private operator. The Project Paper for the first Additional Financing in 2008 had some but not all baseline data and targets for the intermediate indicators.

b. M&E Implementation:
Data was collected to measure achievement of targets. However, some of the indicators either did not achieve targets or did not collect accurate information to determine successful achievement of the target. For example, for the annual gross revenue collection (percent), the baseline was 40% and the target was 50%. The actual value at completion was 35-55% which is too imprecise to determine if the target was achieved or not and if the actual value possibly decreased rather than increased. Additionally, for this indicator the target was an increase in the collection rate from an unspecified baseline.

a. M&E Utilization:
The M&E data was used by the CMWU to plan future investments. For example, the CMWU used the monitoring of water quality to inform donors about the level of service provision to help donors make decisions about future investments (ICR, p. 11). Most indicators were tracked until 12/06/2011, two months prior to project closing. One indicator (volume of water billed to volume of water produced as a percent) was tracked until 12/06/2010 and remained at 62% which was its baseline measure. Since the target was 70% for this indicator the ICR noted this as being "89% achieved." (p. v)

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The project was classified as environmental category B (potentially adverse environmental impacts) at appraisal and an Environmental Impact Assessment and Environmental Mitigation Management Plan were prepared. There was also a social assessment at appraisal which considered the likely impact of raising water tariffs on customers. There were two environmental and social safeguards specialists working on the project. The Environmental Management Plan was implemented by the CMWU. The ICR reported no major adverse environmental or social impact as a result of the project.

b. Fiduciary Compliance:
Procurement. The CMWU was responsible for procurement until 2008 when the management contract was terminated and responsibility for procurement fell to the Project Management Unit. Annual procurement plans were prepared in accordance with Bank guidelines. When access into Gaza was restricted, procurement supervision was done remotely. Procurement performance was rated moderately satisfactory throughout project implementation, despite difficulties bringing in goods following the border closings in 2007-2008.

Disbursement. Audit reports were submitted on a timely basis annually and were unqualified. The ICR reports that "the flow of funds was smooth without major interruptions" (ICR, p. 12).

Financial Management. The Project's financial management team produced timely reports with disbursement and procurement plans. There was no major fiduciary noncompliance according to the ICR.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Unsatisfactory
One of the project's objectives was not achieved. There could have been better evidence on efficiency at project completion. 
Risk to Development Outcome:
High
High
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Unsatisfactory
Both the PA and CMWU were unable to perform satisfactorily after the separation of Gaza. 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Lessons from the ICR
  • "Assistance to utility should be provided based on performance and results" (ICR, p. 21). Continued assistance to the CMWU should have been based on improving its financial standing and improving cost recovery.
  • "Respond to uncertainty with flexibility" (ICR, p. 7). Additional Financing was used at three points during the project to respond to the border closings and the severe reduction in availability of goods and materials needed for the project.
  • The current difficult political and security circumstances justify continuing operational assistance to the utility so that services can continue.

Lesson from IEG
  • As it became evident that the project's objectives were not viable, the project development objectives should have been formally revised or the project should have been cancelled.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR covers all the basic ratings but does not elaborate. This is especially the case for the ratings on Bank performance and monitoring and evaluation. The ICR could have had more discussion of the monitoring indicators.

There were some inconsistencies in the ICR. The dates for revised target values and actual values were confusing since the target values seemed to be calculated after the actual values. There was reference to a PAD even though none exists, only a Technical Annex to the MOP. There was no evidence on progress or decline of indicators over the years, only actual values at project completion. There was no discussion on why the borrower counterpart was not provided. These questions were, however, answered during the interview with the last project TTL. The ICR also reported some of the project output data poorly, showing results of performance indicators as a range that covered improvements to worsening, yet always reporting positive results. The ICR also reported as positive results minute changes in values of indicators that might easily be explained by minor reporting or data errors.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions