Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Local Government Capacity Building Project


  
1. Project Data:   
ICR Review Date Posted:
02/19/2014   
Country:
West Bank & Gaza
PROJ ID:
P116860
Appraisal
Actual
Project Name:
Local Government Capacity Building Project
Project Costs(US $M)
 9.92  9.93
L/C Number:
Loan/Credit (US $M)
 -  -
Sector Board:
Urban Development
Cofinancing (US $M)
 2.47  10.0
Cofinanciers:
Board Approval Date
  03/22/2005
 
 
Closing Date
12/31/2008 06/30/2012
Sector(s):
Sub-national government administration (100%)
Theme(s):
Municipal finance (67% - P) Municipal governance and institution building (33% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Roy Gilbert
Kristin Hallberg Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
To improve local governance and accountability, and thereby foster the efficient and sustainable economic, social and physical development of the urban and rural areas in the parts of West Bank and Gaza under the jurisdiction of the Palestinian Authority (PA), through the capacity building of MOLG (Ministry of Local Government) and LGUs (local government units)" (Denmark Grant Agreement DGA, Schedule 2 p. 11)

A July 31, 2008 Amendment to the DGA retained the original objective, while adding the following details:
1. Improve the accounting, financial management and budgetary control capabilities of MOLG’s finance and budget departments and of the LGUs and the Municipal Development and Lending Fund (MDLF).
2. Strengthening the capacity of MOLG and the LGUs in West Bank and Gaza to effectively guide, manage and monitor the development of the urban areas and be responsive to the needs of its citizens.
3. Enhancing the capacity of MOLG to manage a decentralized local government system. (Amendment to the Trust Fund Administration Agreement para. 5, July 31, 2008).

This ICR Review's assessment, as the ICR's itself, is based upon reviewing achievement of the above objective as whole, while taking into account the additional details formalized by the Amendment.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? No

Date of Board Approval:

c. Components:
1. Financial Management and Accounting Systems Reform (appraisal cost US$4.12m.; actual cost US$6.38m.) including: (i) developing and implementing a detailed specification of a Ministry of Finance (MOF) mandated computerized financial management system based upon the Ministry of Local Government’s needs assessment, with related training; (ii) an inventory of finance-related systems problems in all municipalities; (iii) evaluating and analyzing selected municipalities to develop a detailed specification of a computerized accounting management system for local government units (LGUs); (iv) Methodology/framework and manual to assist municipalities introduce best practices and appropriate finance related computer software; (v) implementation strategy and training for new accounting and financial management systems in selected LGUs and their implementation.
2. Physical Planning and Management Systems Reform (appraisal cost US$2.75m.; actual cost US$1.30m.) including: (i) training and consultants services focused upon: (a) local governments’ proposed physical planning and management legislation and practices, including public participation, their physical and management plans, systems, financing and service delivery, (b)MOLG and LGU development planning and management needs and capacities, (c) implementation strategies, planning and programming guidelines for LGUs in accordance with MOLG and local governments’ laws; (ii) assessment of LGUs technical, economic ad financial performance capacities, to develop a framework and strategy for the concentration of small and unviable LGUs into efficient and sustainable Joint Service Councils (JSCs), including governance structures, management, investment programming, financing and service delivery, and initiating implementation for selected LGUs thereafter.
3. Institutional Development (MOLG) (appraisal cost US$1.84m.; actual cost US$1.44m.) including: (i) developing and implementing capacity programs in project coordination and MOLG budget department support mechanisms; (ii) increasing the capacity of MOLG, including its headquarters and regional offices, to manage a decentralized local government system to focus activities into a manageable framework; (iii) streamlining and improving operational and management procedures of MOLG, including implementation of the appropriate institutional reforms.
4. Project Management and Operating Support (appraisal cost US$0.46m.; actual cost US$0.81m.) to coordinate, monitor and report on the project.)

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Initially, a Danish International Development Agency (DANIDA) trust fund in an amount of US2.5 million was committed to finance the project. A July 31, 2008 Amendment committed an additional US$7.4 million, bringing original total US$9.9 million to be financed entirely by DANIDA (instead of being co-financed by several donors). Unlike most western donors, Denmark continued providing assistance to Gaza after the March 2006 elections that brought Hamas to power. The project was to be fully funded by the DANIDA grant, as part of the multi-donor West Bank and Gaza Trust Fund,with the Bank taking the lead in preparing the project while not providing any financing. By completion, the grant was fully disbursed, so that the final project cost of US$9.9 million was equal to the amount committed. The project’s original closing date of December 31, 2008 was extended three times by three amendments until June 30, 2012. According to the ICR, the main reason for these extensions was the interruption to the project and destruction to Gaza caused by the December 2008 – January 2009 conflict with Israel. The overall result was that a project that was initially intended to be implemented in 4.5 years, was actually only completed in 7.3 years.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Rated: High
The project objective of improving local governance and accountability was and remains highly relevant to the Palestinian Authority’s (PA) priorities for decentralization and improving public sector financial management. These priorities are enshrined in the PA’s 2011-2013 National Development Plan, that also encourages more participation by local communities in public policy in general and in local government budgeting in particular. On the Bank’s side, municipalities feature prominently in the joint Bank-IFC FY12-14 Interim Strategy Note. This Strategy Note highlights the historic role of municipalities in delivering services (p. 17), a key role that they will continue to have (p. 20), and their importance in contributing to national development.

b. Relevance of Design:
Rated: Modest
As presented in the ICR (ICR pp. iii-v) the project’s results framework has three main shortcomings: (i) It addresses only 7 performance indicators compared with the 20 indicators of the PAD’s logframe (PAD pp. 17-21). IEG looked for explanations of the change of narrative in the three legal amendments but could find none there; (ii) it does not explain how output and intermediate outcome results lead to the achievement of the project objectives through an explicit results chain; (iii) it has simplistic and at times unrealistic notions of baseline values, as being simply the absence of what the project component delivered. It reports that the reform of local government physical planning was achieved, for instance, while at the same time affirming that no such physical planning existed at the baseline. This is unrealistic in two ways. First, how can it be possible to reform something that did not exist? Second, no evidence is provided that physical planning was nonexistent in 2005. The PAD itself in 2004 called for reviewing “existing local government physical planning practices” (PAD p. 26) and “strengthening local government physical planning practices” (PAD p. 30). Other than mentioning that the project had ‘design shortcomings’ without specifying what they were, the ICR does not address the relevance of the design of this project (ICR p. 11).


4. Achievement of Objectives (Efficacy) :

PDO: To improve local governance and accountability, and thereby foster the efficient and sustainable economic, social and physical development of the urban and rural areas in West and Gaza. Substantially achieved.
The most important project achievement in this regard was to incorporate regular public participation in local government actions in their municipalities. Thanks to the project, local governments are now responsible for local investments for which they hold public hearings about specific local government plans for urban improvements. More generally, local governments are also now required to hold workshops which the public can attend and call authorities' attention to particular problems and to propose particular investments. This is all very different from the top-down approach taken previously by the MOLG, when it had been in charge of these local investments. There are some shortcomings, however. While the delivery of the project components is reported by the ICR, evidence of the adoption and oversight of the new administrative and financial management procedures needed to demonstrate their impact is thin. Evidence of the recent efficient and sustainable development of West Bank Gaza is not available. As far as West Bank and Gaza development is concerned, World Development Indicators data are only available up to 2005, and therefore cannot provide a time series of economic, social and physical development indicators showing evolution during the 2005-2012 project period.
Intermediate outcomes:
1. Improve the accounting, financial management and budgetary control capabilities of MOLG’s finance and budget departments and of the LGUs and the Municipal Development and Lending Fund (MDLF): Standardized systems of accounting were introduced to all 134 participating municipalities where 2.0 million people, half of West Bank and Gaza’s population lives. As the ICR itself notes, “the impacts upon municipal finances are not yet measurable” (ICR p. 13). The opinion of the Mayor of Nablus conveyed to the ICR mission is offered as evidence of impact (ICR p. 13).
2. Strengthening the capacity of MOLG and the LGUs in West Bank and Gaza to effectively guide, manage and monitor the development of the urban areas and be responsive to the needs of its citizens. The ICR reports that public participation with a voice in planning was introduced for the first time in four local government clusters benefiting 89,291 people, although the report does not specify exactly what measures were adopted. All assisted local governments currently have draft land use plans, according to the ICR, but it does not provide convincing evidence of how many already had such plans at the baseline before the project.
3. Enhancing the capacity of MOLG to manage a decentralized local government system. (Amendment to the Trust Fund Administration Agreement para. 5, July 31, 2008). Through the use of the standardized accounting system introduced by the project, MOLG is able to supervise municipal finances for the first time, according to the ICR (ICR p. 13). Not implementing the project’s departmental action plan for MOLG was a project shortcoming, however.

5. Efficiency:

Rating: Modest: This predominantly technical assistance operation did not lend itself to a cost:benefit analysis to estimate and economic rate of return (ERR) because of the difficulties of quantifying and measuring the benefits. For these reasons the project used cost-effectiveness comparisons to measure its relative efficiency. Thus, the component costs of this project per beneficiary were compared with the costs of similar components implemented in Ghana, Moldova, and Uganda. The result, which the ICR describes as ‘indicative’ points to slightly higher unit costs in the case of this project. The grant itself was made effective within the original time frame pointing to greater efficiency than implied by the ICR remark that “the project took some time to take off’ (ICR p. 7).The ICR’s disbursement profile (ICR p. vi) incorrectly reports that actual disbursements did closely, and efficiently, track planned disbursements, almost exactly. But this is not consistent with the widely reported disbursement delays and the two-year extension of the closing date needed to complete it. In its economic analysis the ICR itself notes that this project’s 7.25 years’ implementation time exceeded the average time of 6.1 years needed to implement similar projects in those countries (ICR p. 29).

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

With high relevance of objectives and modest relevance of design; substantial efficacy; and modest efficiency, the overall outcome rating of this project is Moderately Satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The ICR highlights four actions needed to mitigate the risks to project benefits in the future: (i) more training is needed to ensure the ongoing operation of the project, especially in smaller municipalities; (ii) MOLG’s financial information system still requires further operationalization; (iii) maintenance contract for project software needs to continue; (iv) the project’s revenue management information system still needs adaptations to deal with post-dated checks (ICR p. 10). IEG considers these risks to be significant. Being tied to local governments that have in the past continued their operations even at moments of tension and instability at the macro-level, these project benefits may be at less risk that actions at the national level would be.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
The project was prepared quickly and in an unusual way for the Bank. Initially, it was conceived as being part of the larger Second Emergency Municipal Services Rehabilitation Project (EMSRP II P074594)—rated Moderately Unsatisfactory by IEG.. Having a trust fund contribution below US$2.5 million at the outset meant that it dispensed with standard preparation procedures, such as a concept review, project appraisal document and Board presentation. DANIDA provided US$60,000 however, to finance consultants to prepare the project, only a fraction of the typical cost of a project appraisal, according to IEG’s experience. Speed was of the essence given DANIDA’s time constraints on being able to commit its funds ICR paras. 14, 19). In spite of this, a full project design was not ready in time for the originally estimated loan effectiveness 12 months after approval. The ICR recognizes that the project was under-designed at appraisal, with insufficient attention paid to a municipal reform agenda (ICR p. 7). This became apparent when this project was de-linked from EMSRP-II to become a self-standing operation, subject to the quality standards of a Bank-supported project, following the September 2009 Amendment.During the ongoing political and economic crisis in West Bank an Gaza following the Second Palestinian Intifada of September 2000, Bank assistance had included emergency support as well as backing for longer-term development.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
The Bank fielded active supervision of the project, according to the ICR, that nevertheless does not report the findings and ratings of the missions. Between-mission implementation support was provided by successive country-based task team leaders (TTLs). While supervision could have paid more attention to the original results framework of the PAD, or alternatively proposed a formal restructuring of this project, it did help fully implement a project against considerable odds posed by the volatile macro-situation.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Through its continuous commitment to the project objectives and its support for the reform measures proposed by the project, the Palestinian Authority contributed to the results achieved by this operation. Its performance would have been stronger if it had a clearer strategy for implementing the financial management improvements at the local level that were introduced by the project.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
In October 2005, the PA’s Municipal Development and Lending Fund (MDLF) broadened its role from purely financial intermediation to operate as project implementing agency. In this role, MDLF performed well in dealing with the contractual and fiduciary requirements of the project, in spite of changes in staffing and management.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
M&E design suffered from this project not being prepared as a regular stand-alone Bank-financed operation. According to the ICR ‘the M&E framework design did not follow the standard PAD format, and does not adequately distinguish the different links in the result chain (Activities, outputs and outcomes) leading to a focus on outputs rather than outcomes’ (ICR p. 8). While IEG concurs with the ICR that the PAD’s Logframe and Key Performance Indicators (PAD Annex 1 pp. 18-21) did focus predominantly on project outputs, it did include the following outcome indicator ‘LG Laws revised, granting significant autonomy to LGUs and providing for increased public participation and accountability’ that is disregarded by the ICR (pp. iii-v, 8), because it ‘does not relate to any project outputs. Finding fault in the M&E design is fair in this case, but the remedy should be to disregard the irrelevant output indicators, not the outcome indicator itself.

b. M&E Implementation:
M&E was only implemented for the purposes of monitoring the implementation of the project itself. For that routine project progress reports were assembled, but they did not address the extent to which project implementation was contributing to the achievement of the project objective.The ICR reports that changes were made to the original results framework and performance indicators without seeking to go through a Bank process of restructuring that is required in these circumstances (ICR p. 9).

a. M&E Utilization:
The ICR reports that M&E information was satisfactory, but does not explain if or how this information was used to guide the project toward achieving its objectives.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
As primarily a technical assistance operation, this project was rated Category C as far as Bank environmental and social safeguards are concerned. The ICR reports that there were no negative environmental or social impacts

b. Fiduciary Compliance:
In general, the project was in compliance. Financial reporting was regular and on time, as were financial audits that came with unqualified opinions on the project’s finances and financial management. There was one exception, however. A US$61,000 contract for the purchase of computers did not go to the lowest bidder, who was unable to deliver by a certain date that had not been specified in the bidding documents. This would normally have been treated as a case of mis-procurement, but the Bank did not call for it on the grounds of what the ICR calls a ‘genuine misunderstanding of the guidelines’ (ICR p. 10). When there is mis-procurement, it should be reported even though it was based upon a misunderstanding.

c. Unintended Impacts (positive or negative):
none

d. Other:
none



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
Although quality at entry was found to be moderately unsatisfactory, the rating guidelines indicate that when the overall project outcome is in the satisfactory range as in this case, then bank performance is rounded up to the higher of its two sub-ratings. 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Implementing improved financial management in local governments is an essential element of improving local governance and oversight, but it must be resilient to management changes, technical challenges, and possible contractual disputes during implementation.
Full project appraisal to ensure high quality at entry is essential for a satisfactory outcome. The decision not to carry out a full design in this case resulted in some activities not being properly appraised. Even those projects which do not formally require full appraisal and documentation could well benefit from going the extra mile by documenting the project fully in a standard format, to make sure that lessons learned are incorporated and risk is properly evaluated.
Impact of changes in Bank Policy for projects under implementation should be not be underestimated and should be well communicated to Clients.During the seven year life time of the project, Bank policies and procedures changed several times. They included instituting a new Bank reporting system. While such changes can be expected during the course of a project, they should be communicated to clients beyond the project level.
Operating in conflict environments necessitates a high level and creativity of supervision, especially for projects aimed at institutional and financial reform. While this operation achieved positive, but limited results, its financial reform element was in jeopardy for extended periods of time, due to contractual disputes aggravated by volatile macro-context. These were only overcome thanks to additional efforts by all parties,including strong engagement by the Bank supervision team and management and use of video conferencing to communicate with staff and beneficiaries in Gaza
When needed as it often can be in conflict areas, project restructuring should be encouraged, with clear instructions and guidance to task teams. In hindsight in this case, it was clear that restructuring was overdue without their being any formal reason for not pursuing it. The lack of restructuring, notwithstanding the minor amendments of the last two years of implementation, allowed the project to drift away from a results framework and its M&E that could have guided the operation toward a more successful result.
Innovation in important areas such as local government financial management needs to take into account the real technical capabilities locally to implement the proposed changes.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR does a good job in describing the preparation and implementation of this project in very challenging circumstances. As a self-evaluation of project performance, there are a number of shortcomings that, among other things, lead the ICR give higher performance ratings than warranted by the evidence provided. These shortcomings include: (i) a disbursement profile (ICR p. vi) that incorrectly reports the levels of actual disbursements and wrongly indicates that they closely tracking planned disbursements when considerable delays are widely reported elsewhere; (ii) weak assessment of the project’s results framework by not making the results chain between component outputs and achieving the objectives clear, with default baseline data being the absence of the project component. The ICR could have explained more clearly that the project’s original results framework at appraisal (PAD pp. 18-21) had not been taken into account as it had not been adequately prepared; (iii) Incorrectly inverting the relationship between higher level final outcome indicators and lower level output indicators, when it discarded the PAD outcome indicator of local government laws revised, because it ‘does not relate to any project outputs’ (ICR p. 8); (iv) inconsistent reporting of project component costs between list of components (ICR pp. 3-4) and Annex 1 Table (ICR p. 21).

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions