Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Emergency Municipal Services Rehabilitation Project II


  
1. Project Data:   
ICR Review Date Posted:
06/28/2013   
Country:
West Bank & Gaza
PROJ ID:
P074594
Appraisal
Actual
Project Name:
Emergency Municipal Services Rehabilitation Project II
Project Costs(US $M)
 40.2  70.28
L/C Number:
Loan/Credit (US $M)
 10  12.58
Sector Board:
Urban Development
Cofinancing (US $M)
 30.2  57.7
Cofinanciers:
France, Netherlands, KfW, SIDA and DANIDA
Board Approval Date
  12/19/2006
 
 
Closing Date
06/30/2009 06/30/2011
Sector(s):
General water sanitation and flood protection sector (35%), Power (23%), Roads and highways (18%), Other social services (12%), General public administration sector (12%)
Theme(s):
Urban services and housing for the poor (25% - P) Other social protection and risk management (25% - P) Municipal finance (24% - P) Pollution management and environmental health (13% - S) Social safety nets (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Midori Makino
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
As per the Trust Fund Grant Agreement, Schedule 1, the project's development objectives were to:

(a) provide assistance for mitigating further deterioration in the delivery of essential municipal services in the West Bank and Gaza;
(b) create temporary job opportunities at the local level; and
(c) promote pilot initiatives that foster collaboration among local governments and NGOs and promote municipal cost recovery.

The appraisal document, in this case the technical annex to the trust fund grant, formulated the project objectives as follows:
(i) help mitigate further deterioration in the delivery of essential municipal services, including solid waste collection, street lighting, provision of water, wastewater and electricity services (where such services are provided by municipalities), and street cleaning, among other services, through rehabilitation of damaged or deteriorating infrastructure and the provision of non-wage budgetary support for direct service cost inputs;
(ii) create temporary job opportunities at the local level through the launching of labor-intensive employment generation schemes, proportionally allocated on the basis of a combined unemployment/poverty index; and
(iii) pilot innovative initiatives that foster Local Government-NGO collaboration in response to urgent community needs and assist municipalities in recovering costs for the provision of services.

This ICR review uses the statement of objectives in the Trust Fund Grant Agreement because it is more monitorable.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
(i) Emergency municipal services assistance grants (appraisal cost US$29.7 million; actual cost for Bank financed portion US$ 8 million, actual cost for the remaining project component not available) including rehabilitation and reconstruction of damaged infrastructure and facilities, supply of necessary equipment, and financing of non-wage recurrent costs of municipal services over a two year period. Priority areas were for water and wastewater services, solid waste, road rehabilitation and maintenance, and electricity.
(ii) Labor-intensive employment generation (appraisal cost US$6.1 million; actual cost for Bank financed portion US$3.1 million, actual cost for the remaining project component not available) including labor-intensive sub-projects--with a minimum of 50% labor content--identified and implemented by municipalities in collaboration with local community groups.
(iii) Municipal Development and Lending Fund innovation window (appraisal cost US$1.2 million; actual cost for Bank financed portion US$ 1.2 million, actual cost for the remaining project component not available) including piloting pre-paid metering systems in three municipalities, and piloting local government/NGO collaboration for emergency and other joint programming purposes, such as public awareness campaigns to promote conservation, public health, environmental management.
(iv) Project management costs (appraisal cost US$3.2 million; actual cost for Bank financed portion US$0.7 million, actual cost for the remaining project component not available) based upon a Municipal Development and Lending Fund (MDLF) management fee and technical assistance to establish three regional extension services.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost and Financing. According to the technical annex to the Trust Fund Grant the total project cost at appraisal was $40.2 million, of which $10 million was to be provided through a Bank grant and the remaining to be provided by other donors. In July 2009 the Bank provided additional funding of US$3.0 million for further rehabilitation in Gaza of infrastructure that had been damaged in Israeli air attacks of December 2008.

At appraisal, the French Development Agency ($16.57 million actual) and the Government of the Netherlands ($6.17 million actual) had committed to contribute to the project costs. During project implementation, additional donor funding was provided through German Development Bank (KfW) in the amount of $20.71, Swedish SIDA in the amount of $8.62 million, and Danish DANIDA in the amount of $5.22 million. This additional funding has contributed to the revised total project cost of $70.28 million. ICR does not provide data on actual financing provided by these donors for each project component.

Borrower contribution. There was no Borrower contribution planned and none materialized.

Dates. The project's original closing date was June 2009, but it was extended first to December 2009 to enable the completion of the parking meter pilot, and for the second time to June 2011 to enable the implementation of the additional US$3 million funding for Gaza.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Relevance of objectives is rated Substantial. Objective (a) of mitigating further deterioration of municipal services is highly relevant to the priorities of the current Palestinian National Development Plan (PNDP) 2011-2013 that allocates 17% of its budget to strengthening local governance and service provision. It is also highly relevant to the Bank's 2008-2010 Interim Strategy for West Bank Gaza (IS-WBG 2008-2010), that treats municipal services as one of its two main pillars, assistance for which is coordinated through a special Municipal Sector Working Group of donors.
Objective (b) creating temporary job opportunities is relevant to the Bank’s 2008-2010 Interim Strategy that links the acute unemployment issue with high poverty levels and outlines the Palestinian Authority's response as "to provide for the rapidly growing population by increasing social transfers and becoming the employer of last resort (ISN p.8). The objective to create job opportunities remain modestly relevant to the 2011-2013 Development Plan which now focuses upon permanent job creation.
Objective (c) of fostering local government collaboration with NGOs, and promoting municipal cost recovery is highly relevant to the priority that the 2011-2013 National Development Plan gives to service provision by both municipalities and NGOs. It is also highly relevant to the Bank’s Interim Strategy with its focus on cost recovery from local services.

The project objectives were based upon a realistic assessment of the volatile country conditions of West Bank Gaza arising from unpredictable closures, military activities and the intifada. The ICR candidly acknowledges that project appraisal in 2006 was at a time of acute crisis.

b. Relevance of Design:
Relevance of project design is rated Substantial. The development objectives were clear and measurable. The activities included in the project components were consistent with the objectives. Thus grants for the emergency rehabilitation and reconstruction of damaged infrastructure would be expected to bring municipal services back on line and prevent their further deterioration, as intended. Project financed sub-projects were specifically designed to use labor inputs intensively to achieve the second objective of creating temporary job opportunities. Activities planned under the Municipal Development and Lending Fund innovation window could be linked to the municipal cost recovery objective. Pilot sub-projects were included in the components to foster collaboration between local government and NGOs, as reflected in the the project objective.


4. Achievement of Objectives (Efficacy) :

1. Mitigating further deterioration in the delivery of municipal services. Modest.

Outputs (physical output targets were not set at appraisal because the project was designed to be demand driven):

Water supply and sanitation:
  • 3.9km of water networks rehabilitated.
  • 903 water meters installed.
  • 1.69km of wastewater networks rehabilitated.

Solid waste:
  • 447 solid waste containers installed.
  • 2 tractors and 3 trailers installed.

Electricity: 9km of electrical networks rehabilitated.

Roads:
  • 72.5km paved
  • 10km sidewalks constructed or rehabilitated.
  • 4.6km of street lighting networks rehabilitated.
  • 2,973 street lighting units installed.
  • 809 lm of culverts and drainage rehabilitated.
  • 240 lm hand rails constructed or rehabilitated.
  • 630 lm of retaining walls constructed.
  • street roller asphalt cutter, excavator, and concrete mixer installed.

Public/municipal buildings:
  • 3,958 m2 of public buildings constructed or rehabilitated.
  • 1,430 m2 of public facilities constructed or rehabilitated.
Budgetary support to purchase spare parts, pay for dumping site fees, service tools, labor insurance, vehicles insurance, and fuel to maintain basic services.

Outcomes: Through the project, 132 municipalities where 70 percent of the West Bank and Gaza's population live benefited from project assistance. 81 percent of respondents to a beneficiary survey of residents of participating municipalities reported that they were satisfied with the level of services received, against the target of 70 percent.

While the large share of the satisfied beneficiaries indicates that the municipal services are at an acceptable level for them, the level of achievement in "mitigating further deterioration of municipal services" is unclear for the following reasons. First, the technical annex prepared at appraisal does not provide quantified evidence that the level of municipal services that it aimed to halt although it describes the deteriorating municipal services as one of the key sector issues. Second, the ICR does not provide quantified evidence that the level of municipal services had not deteriorated further. Third, the ICR reports only the Bank's financing which represented less than one fifth of the project’s total costs. Fourth, there is no baseline of beneficiary satisfaction, and the evidence from the beneficiary survey is based on interviews with 614 of the project’s 2.8 million beneficiaries. While it is not clear how the 614 were chosen, the ICR provides the demographic breakdown of respondents and the split between West Bank and Gaza, indicating that the respondents are representative across the territory.

2. Creating temporary job opportunities. Substantial.

Outputs: 69.58 percent labor content for project financed works was achieved against the target of 50 percent. This result was achieved through creation of temporary job opportunities at the local level through labor intensive employment generation schemes, proportionally allocated on the basis of the Municipal Development Fund unemployment index. The sub-projects were identified and implemented by Municipalities in coordination with locally-based community groups. Sub-projects varied by sector, and included agriculture, NGO-based services, and other labor intensive activities.

Outcome: Temporary job opportunities were created for the beneficiaries in the municipal area. 167,000 person days of employment exceeding the project’s target of 140,000 person days, in addition to the 3,304 person days of indirect temporary employment.

3. Fostering local government/NGO collaboration and promoting municipal cost recovery. Modest

Outputs:

Pilots for local Government NGO collaboration for emergency and other joint programming purposes were carried out to deliver municipal services more effectively. Ten pilots were implemented out of the 33 proposals, including a community center developed in one municipality in Gaza; women’s training center constructed in Ajanta municipality in the southern West Bank; and a park and community center developed in Bait Shout municipality, also in the southern West Bank. The total number of beneficiaries from the NGO subprojects was 22,180 in the West Bank and 18,387 in Gaza. However, the sub-component on Municipal Development Fund Innovation Window suffered from lack of clarity on linkages to the emergency program, ownership and implementation mechanisms.

Prepaid electric metering systems which was planned to be implemented for improving municipal service cost recovery was replaced by a parking meter scheme.

Outcome:

While positive outcomes were observed with regard to social development and benefits for women in certain NGO pilot activities, the municipal/NGO collaboration ceased once the facilities were handed over to NGOs for operation and therefore the objective of fostering local government/NGO collaboration was not fully achieved. No data are provided on the revenues derived from the parking meters and the actual number of meters installed is unknown. It is unclear if there has been any cost recovery to date. The meters have been installed, but the ICR refers to "issues of enforcement".


5. Efficiency:

The project was prepared under emergency recovery assistance procedures (OP/BP 8.50); there were time limitations and limited availability of economic data that prevented an economic analysis at appraisal, other than the description of potential areas of economic benefits.

Cost efficiency analysis done for the disbursements related to maintenance of specific segments in the water, electricity, and sewer networks revealed that these were more cost efficient than the alternative of reinvesting in new networks. Similarly, for the non-capital expenditures associated with road rehabilitation, solid waste management, and public facilities it was verified that Municipal Development and Lending Fund's procurement and execution system met cost efficiency standards.

At closure, the ICR estimates economic rates of return of of 25.8%, 37.8% and 14.8% respectively for investments of US$1.5 million in roads, US$0.14 million in solid waste and US$0.52 in municipal buildings. However, these refer only to the Bank financed portion, the total of which amounts to 4% of final project costs. The ICR does not report on the efficiency of the remaining 96% of the project resources spent.

Overall efficiency is rated Modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
Yes
23.9%
4%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Both the relevance of objectives and that of design were substantial. However, there were significant shortcomings in efficacy with modest achievement of two out of three objectives – there is a lack of evidence as to whether further deterioration of municipal services has been mitigated, or effective municipal/NGO collaboration, and of cost recovery. Given the lack of evidence also on the efficiency with which 96% of project resources were used, efficiency is rated modest.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

Beyond the high risks presented by the precarious political and security situation in West Bank Gaza, there are other factors that increase the risk to development outcome. Not knowing the full costs of each project component poses the risk of loss of financial control over project expenditures. Financial risks to cost recovery are high when revenues due--such as from parking meters--are not collected. While the objective of fostering local government/NGO collaboration was not fully achieved under the project, the region has subsequently reported that all NGOs that participated in the project "would prefer a repeat of such development projects in the future." and the lessons from the pilot informed future NGO programs by the Bank. Continued government commitment and ownership would depend in large part on the continuity of support from external partners.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
Prior preparation for a larger municipal development project that was cancelled at the request of the Palestinian Authority helped ensure the quality-at-entry of the current operation in four ways. First, it provided a firm basis for the execution of an emergency operation. Second, its cancellation made clear the operational constraints arising from the emergency, making it easier for the Bank to assess the risks up-front. Third, it allowed this emergency project to give far more attention to institutional and governance issues than would typically have been the case for a rapid response operation. Fourth, it resulted in a more significant deployment of Bank staff and other inputs than would be usual for this size of operation. However, the ICR acknowledges that the project design was nonetheless weakened by including too many output indicators for practical monitoring, and by insufficient Borrower ownership of the innovative pilot schemes proposed for the project. M&E design was weak, the design relevance was modest, and there were no provisions for fully taking account of parallel financing and assistance. The project did not take account of the “volatility and instability in contexts such as West Bank and Gaza's” as highlighted in the lesson's section.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:
Bank supervision was challenging but the Bank team made efforts by being flexible, responsible, and supportive to the client needs. Even though access to Gaza was restricted, the Bank carried out supervision missions through video conferencing and managed to provide necessary support to implement the project. However, there were major shortcomings: (i) only taking limited corrective actions to improve the weakness in the M&E design during project supervision; (ii) not reporting on the project investments supported by other financiers that constituted the bulk of the project effort; and (iii) not reporting explicitly on fiduciary and safeguard compliance by all donors.

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
In spite of the internal divisions between the Palestinian Authority (in the West Bank) and Hamas (in Gaza), the Government (Ministry of Finance and Ministry of Local Government) showed strong support for the project in both territories. Both took the necessary steps to ensure that Bank and project requirements were met on a timely basis (particularly ratifying the environmental management plan and executing subsidiary agreements), and complied with all legal agreements pertaining to the project. A remaining issue that the Government still had to address at closure was to formalize the relationship between the Municipal Development and Lending Fund and the Ministry of Local Government.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
The Municipal Development and Lending Fund, an autonomous juridical entity established pursuant to the Council of Ministers Decree dated October 20, 2005, was the implementing agency. The Municipal Development and Lending Fund was responsible for the overall supervision of the project and was authorized to execute payments to municipalities and contractors. It also provided assistance to the 132 participating municipalities to improve implementation of the sub-projects. The Municipal Development and Lending Fund was effective in; (i) developing and implementing more transparent rules for the allocation of funds to the sector; (ii) helping municipalities to prepare projects, (iii) persuading them to comply with their legal obligation to submit budgets on a timely basis; (iv) building a solid municipal database; and (v) resolving the initial problems encountered with local technical consultants, as well as those stemming from the depreciation of the US dollar, and contractors’ strike problems. M&E, however, which was the Fund's responsibility, was weak for the whole project (including the portions that were funded by other donors).

Project start up was slow, due in part to weak procurement management at the local level. The ICR reports that the municipalities showed considerable improvements over time and were able to take the project forward without any report or allegation of mis-procurement. According to the ICR, municipalities also strengthened their financial controls and reporting.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The Technical Annex to the Memorandum and Recommendation of the President discussed the key project performance indicators and mentioned that the Project Operations Manual describes the process of monitoring the outcomes and outputs of the project. However, there were shortfalls in the results framework as the indicators chosen included sector indicators, such as inter-governmental fiscal transfers and public disclosure of municipal budgets, that were not directly linked to the project objectives. Other indicators did not clearly specify to what extent they reflected the objectives, and relied heavily on beneficiary surveys without baseline data. For example, the assessment of the deterioration in the delivery of essential municipal services should have included separate indicators related to the level of key municipal services such as solid waste collection or roads which the project supported, rather than on one customer satisfaction indicator for all services. The most important shortcoming was not establishing a framework in the Municipal Development and Lending Fund, which was in charge of maintaining all project data in a database at appraisal, for systematically collecting the information from the components that were funded by other donors.

b. M&E Implementation:
The ICR states that the mid term review had made some revisions to the M&E design and the beneficiary survey was conducted to partially mitigate the weakness of the M&E system, but the most important shortcoming related to the framework for the Municipal Development and Lending Fund was not strengthened. Although it reports that a considerable body of data was collected during implementation, this does not appear to have been used for evaluation purposes.

a. M&E Utilization:
The ICR also acknowledges that the project did not compile information and assessments of performance that could be used to steer the operation toward greater success. In this regard, the ICR reports that the results of beneficiary assessments came too late to provide feedback on project performance during implementation. Over-reliance on such assessments, which are normally used to gauge the impact of the project, as an M&E tool illustrates the overall weakness in this project's approach to M&E. In addition, the ICR does not report on the achievements or the cost information of the activities funded by other donors who contributed to the project, indicating weak M&E utilization.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
At appraisal, the project was assigned Environmental Category "B" and tow safeguards policies were triggered: Environmental Assessment (O.P 4.01) and Pest Management. An environmental management plan was prepared, During implementation, an environmental specialist was hired specifically to help municipalities implement the plan. Safeguard requirements that might arise through involuntary resettlement were avoided by excluding any sub-projects that might have required land acquisition. The ICR does not contain an explicit statement about safeguard compliance by all donors, but states that "there were no significant safeguard compliance issues at any time during the project" (ICR p. 5).

b. Fiduciary Compliance:
The ICR does not state specifically that there was compliance with the Bank's fiduciary policies, but reports that financial reporting was regular and all audit reports were unqualified. Procurement management, especially at the municipal level was weak at the outset, but improved during implementation thanks to the assistance provided through the project. No case of misprocurement was reported. ICR does not report on fiduciary compliance by all donors that contributed to the project.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Unsatisfactory
Both the relevance of objectives and that of design were substantial. However, there were significant shortcomings in efficacy with modest achievement of two out of three objectives – there is a lack of evidence as to whether further deterioration of municipal services has been mitigated, of effective municipal/NGO collaboration, and of cost recovery. Given the lack of evidence also on the efficiency with which 96% of project resources were used, efficiency is rated modest. 
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Satisfactory
Moderately Unsatisfactory
Quality at Entry was moderately satisfactory. However, Quality of Supervision is rated moderately unsatisfactory because of: (i) only taking limited corrective actions to improve the weakness in the M&E design during project supervision; (ii) not reporting on the project investments supported by other financiers that constituted the bulk of the project effort; and (iii) not reporting explicitly on fiduciary and safeguard compliance by other donors. The Harmonized Evaluation criteria result in the moderately unsatisfactory overall Bank Performance rating, factoring in the moderately unsatisfactory outcome rating.  
Borrower Performance:
Satisfactory
Moderately Satisfactory
Reflecting moderate shortcomings at the implementing agency level. 
Quality of ICR:
 
Unsatisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR acknowledges that not all of its nine lessons derive from the project experience. Here, the following ICR lessons that mostly do so are summarized and paraphrased.
  • If experimental pilot sub-projects--such as parking meters for towns under this operation--are included in emergency project of this kind, they will likely distract attention from the rapid response needed, especially when they have little to do with the emergency at the outset.
  • Especially when political consensus is weak (whether there is open conflict or not), it is essential for project design to appeal to the fractured interests of different stakeholders. Trying to incorporate implementation arrangements that are deliberately non-partisan, and are clearly seen as such, as the Municipal Development and Lending Fund became under this project, can be appropriate.
Lesson from discussions with the project team is:
  • Volatility and instability in contexts such as West Bank and Gaza's should be factored into the project design so that half-expected difficulties do not disrupt implementation and results. In this case, the design focused upon using recycled building material so that project construction would not be hostage to the inevitable closures and boycotts that arise.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR was provided evidence on outputs financed by the Bank, which constituted only 18% of the actual project costs. The ICR does not provide sufficient evidence on outcomes. From the project team, IEG learned that the Bank interactions with other donors had been intensive and that all their cost information was available to the Bank, but that the task team was inappropriately advised that it was not necessary to include it in the ICR. This exemption to exclude reporting of other donor funding was not documented in the ICR. Some lessons appear contradictory, such as the lesson about avoiding the use of pilot components in emergency projects which contradicts another one calling for support for innovation (for which pilots can be useful). The ICR does not contain an explicit statement about safeguard and fiduciary compliance by all donors.

a. Quality of ICR Rating: Unsatisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions