Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Palestinian Ngo Project Iii


  
1. Project Data:   
ICR Review Date Posted:
10/03/2012   
Country:
West Bank & Gaza
PROJ ID:
P096777
Appraisal
Actual
Project Name:
Palestinian Ngo Project Iii
Project Costs(US $M)
 17.6  20.45
L/C Number:
Loan/Credit (US $M)
 10.0  12.85
Sector Board:
Social Development
Cofinancing (US $M)
 7.6  7.6
Cofinanciers:
Agence Francaise de Developpement
Board Approval Date
  12/19/2006
 
 
Closing Date
12/31/2009 09/30/2011
Sector(s):
Other social services (84%), Health (16%)
Theme(s):
Participation and civic engagement (29% - P) Social safety nets (29% - P) Health system performance (14% - S) Other social protection and risk management (14% - S) Improving labor markets (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judith Hahn Gaubatz
Soniya Carvalho Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
According to the Project Appraisal Document (PAD, page 4) and the Trust Fund Grant Agreement (TFGA, page 6), the objective of the project was:

  • to provide social services to those who are poor, vulnerable, or affected by the deteriorating socioeconomic conditions by establishing an effective mechanism to improve the quality and sustainability of NGO social service delivery.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1: NGO Grants for Social Service Delivery (Appraisal: US$14.22 million; Actual: US$16.72 million): This component aimed to provide different types of sub-grants to NGOs, including for poor and vulnerable groups. The types of grants focused on sectors and themes in which NGOs had a comparative advantage. These included: (i) Empowerment grants for providing basic social services in response to dire local needs; (ii) Mentoring Partnership grants for service delivery by small community-based NGOs in partnership with experienced NGOs; (iii) Specialized Health grants for addressing the current crisis in health service provision; (iv) NGO-Local Government Partnership grants for pilot activities jointly implemented by municipalities and NGOs; and (v) Emergency grants for short-term job creation in communities impacted by the ongoing economic crisis. All NGOs were expected to contribute at least five percent towards the sub-project costs.

2: NGO Sector Development (Appraisal: US$1.1 million; Actual: US$1.05 million): This component aimed to further develop the NGO sector to become more responsive, transparent, and accountable to local communities. Activities included: development of an NGO Code of Conduct; promotion of collaboration within the sector through information exchange; and strategic policy and research.

3. Institutional Development of the NGO Development Center (Appraisal: US$2.28 million; Actual: US$2.69 million): This component aimed to support organizational effectiveness of the NGO Development Center (NDC) in the provision of technical assistance and grants to NGOs. Activities included: establishment of NDC operations; resource mobilization and communication; and development of an outcome-based monitoring and evaluation system.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost:

  • The actual project cost was US$20.45 million. This was higher than the original appraised cost of US$17.6 million, but equivalent to the revised appraised cost of US$20.6 million (due to additional financing).

Financing:
  • In July 2009, the Bank, at the request of the Palestinian Authority (PA), approved Additional Financing in the amount of US$3.0 million for social service support in Gaza, due to the 2008/09 security conflict which had severely disrupted activities. US$2.64 million was allocated to Component 1 for NGOs based in Gaza; US$0.36 million was allocated to Component 3 for M&E activities, technical assistance to NGOs, and supervision and audits.
  • Agence Francaise de Developpement (AfD) provided US$7.6 million in co-financing, as planned. The amount in its entirety was used for provision of Emergency grants under Component 1.

Borrower Contribution:
  • There was no planned Borrower contribution.

Dates:
  • In 2008/09, the security situation in Gaza deteriorated significantly, causing border closures and destruction of infrastructure, institutions, and social services.
  • In July 2009, as noted previously, the Bank approved Additional Financing of US$3.0 million due to the effects of the conflict in Gaza.
  • In September 2009, the project closing date was extended from December 2009 to September 2011 to allow remaining activities to be fully implemented. A reallocation of US$335,000 within Component 1 - from Health and Mentoring grant categories to the Empowerment grant category - was also approved, as well as the Bank's new standard conditions regarding anti-corruption guidelines.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial. The emergency provision of basic social services was a substantially relevant objective, as a large proportion of the population was being impacted by the ongoing security and fiscal crises. The strengthening of NGOs was also relevant, given their role in filling an important gap in service delivery as well as providing a means for reaching vulnerable groups. The Bank's Interim Strategy Note (ISN), which was updated for FY12-14 just at the time when the ICR was completed, identifies ensuring services to citizens as the first key pillar, including making services more responsive to users' needs. The Palestinian National Development Plan (2011-2013) included support and empowerment of the poor and other vulnerable citizens as a main strategic objective in the social sector, as well as strengthening partnerships with local communities as part of the Government's efforts at increased accountability and responsiveness.

b. Relevance of Design:
Substantial. Project outputs were likely to lead to intended outcomes within the planned time frame. The project design provided for direct financing of NGO activities through grants due to the emergency context, including for NGOs that work with specific vulnerable groups or in targeted geographic locations. It also supported overall NGO sector and institutional development by institutionalizing accumulated experience. Building on the achievements of the Bank's two preceding NGO projects (Palestine NGO I: 1997-2003 and Palestine NGO II: 2001-2006), this third NGO project aimed to establish an NGO Development Center which would take responsibility for further NGO sector development.


4. Achievement of Objectives (Efficacy) :

Objective: To provide social services to those who are poor, vulnerable, or affected by the deteriorating socioeconomic conditions: Substantial

Outputs
  • Provision of 50 social service delivery grants to NGOs, of which there were 29 empowerment grants (US$2.63 million), 8 mentoring grants (US$2.2 million), 7 specialized health grants (US$1.6 million), and 6 local government grants. These grants were estimated to have reached 268,538 beneficiaries.
  • Of the 29 empowerment grants provided, the ICR (Annex 5) provides detailed outputs for 5 grantees, including 12,000 textiles produced, 80 students participating in annual music summer camps, 13 kindergarten programs, 4 classrooms for children with learning disabilities, and 846 hours of computer and library lessons.
  • Of the 8 mentoring grants provided, the ICR (Annex 5) provides detailed outputs for 3 grantees, including provision of veterinary and extension services to farmers; educational training for children with visual impairments; and equipping of one general health practitioner clinic.
  • Of the 7 health grants provided, the ICR (Annex 5) provides detailed outputs for 4 grantees, including establishment of a diabetes center, a cancer diagnostic center, an advanced neurosurgery department, and a pediatric cardiology department.
  • Provision of 51 emergency job-creation grants to NGOs for short-term employment generation. These grants were solely funded by co-financing from Agence Francaise de Developpement (AfD). These grants created 143,300 working man-days, achieving the target of 140,000, and represented 35% of labor content in all construction activities. The grants focused on the agricultural sector, producing 6,119 donums (unit of land area that can be plowed in a day) rehabilitated and planted; 989 greenhouses established or rehabilitated; 105 agricultural wells excavated; and 14 km of roads opened.

Outcomes on targeting
  • 40% of empowerment grant beneficiaries lived in localities that were isolated by the barrier walls, surpassing the target of 20%.
  • 75% of mentoring grant beneficiaries were from targeted vulnerable groups, including youth, women-headed households, elderly, children, and people with special needs, surpassing the target of 50%. 78% were from the targeted geographic areas, surpassing the target of 60%.
  • 48% of specialized health grant beneficiaries were from targeted vulnerable groups or poor households, nearly achieving the target of 50%.
  • 19 out of the 50 approved NGO grants reached beneficiaries in the villages of Hebron, which have the highest poverty rates.
  • 10 out of the 50 approved NGO grants specifically targeted women.


The objective was to be achieved by establishing an effective mechanism to:

Improve the quality of NGO social service delivery

Outputs
  • Registration of the NGO Development Center (NDC) as an independent NGO. The NDC supported capacity building efforts in the areas of good governance, transparency, accountability, and service performance standards. The ICR (page 15) reports that the NDC improved its procurement capacity and performance during the project period and created an internal audit function.
  • Development and publication of the Code of Conduct for NGOs (outlining principles of governance, accountability and transparency). At the time of ICR preparation, 570 NGOs (out of 2,950 existing NGOs) had signed the Code and 45% of those NGOs were implementing the Code standards. Public disclosure of financial statements increased from 14% to 60%.
  • Establishment of a computerized web-based and outcome-based project monitoring system, staffed by a full time M&E Officer. The system is capable of tracking implementation progress and generating data to prepare technical and financial reports in a timely manner.
  • Provision of technical assistance to four NGO umbrella networks (which represent 90% of all NGOs) to promote knowledge sharing, collaboration and coordination, development of best practices, and advocacy for the NGO sector.

Outcomes
  • From an institutional standpoint, quality may have been improved due to the creation of the independent NDC which aimed to institutionalize past experience in the NGO sector. The NDC supported capacity building on longer-term governance issues, including the implementation of the Code of Conduct.
  • According to the beneficiary impact assessment conducted by the Borrower,* 87.7% of all beneficiaries interviewed indicated services were highly beneficial. 65% of Empowerment grant beneficiaries rated quality of services as high, achieving the target of 60%. 68% of Mentoring grant beneficiaries rated quality of services as high, achieving the target of 60%.
  • According to an ICR field assessment and capacity building assessment, all Specialized Health grant projects were technically sound and achieved their objectives.
  • 70% of sub-projects were implemented according to community demand, surpassing the target of 30%.

However,
  • Specific information on the content of all social services provided by the grantees is not provided (aside from Annex 5 of the ICR which reports outputs for 13 of the 50 grantees covered by the beneficiary survey - see above). There is inadequate information to assess improvements in quality.


Improve the sustainability of NGO social service delivery

Outputs
  • Establishment of the NGO Development Center that mobilized and managed funds for NGOs (US$17.22 million funds raised, surpassing the target of US$2.0 million).
  • Provision of technical assistance to four NGO umbrella networks (which represent 90% of all NGOs) to promote collaboration and coordination, development of best practices, and advocacy for the NGO sector.
  • Completion of three studies: promoting areas of cooperation between NGOs and the private sector; documenting and analyzing donor financing; and a strategy for the NGO sector.

Outcomes
The PAD Results Framework (Annex 3) identifies three indicators for the sustainability objective: mobilization of resources for financing NGO services (financial sustainability); percentage of sub-projects with at least 10% NGO contribution to costs (financial sustainability and ownership); and percentage of NGOs that continue to sustain operations one year after grant completion (operational sustainability and ownership).
  • According to the beneficiary survey conducted by the ICR team,** most beneficiaries interviewed raised concerns about sustainability and requested additional support.
  • A number of NGOs were receiving community contributions to supplement the grant funding. 44% of Empowerment grant recipients provided at least 10% contribution (cash or in-kind) towards project costs, surpassing the target of 30%. According to the Borrower's ICR (ICR, page 47), "the vast majority of the NGOs interviewed by the ICR consultant reported that they succeeded in securing financial or in-kind support to their projects."
  • It was not yet clear whether NGOs that received funding were able to sustain operations at least one year after the grant was completed. Three Empowerment grant recipients that had completed their projects one year prior to the time of ICR preparation were still in operation. However, all other sub-projects had not yet been closed for a year, and therefore could not be assessed.


* According to Annex 5 of the ICR, the Borrower conducted a Beneficiary Impact Assessment in May 2010, covering 35 out of 134 projects financed by NDC. There are no further details provided on the methodology.
** According to pages 17 and 34 of the ICR, the ICR team conducted a Beneficiary Survey in October 2011, covering a random selected sample of 13 projects financed by NDC. The survey included quantitative performance indicators, photographs of post-project physical infrastructure, and qualitative opinion interviews

5. Efficiency:

Modest. For Component 1, the total amount of funding for sub-projects (not including the Emergency grants co-financed by AfD) was US$9.12 million, which was estimated to have reached 268,538 beneficiaries directly. However, there was insufficient information to assess efficiency ( i.e. analysis of unit cost per service provided). For Components 2 and 3 which supported the NGO Development Center (NDC), the project contributed to efficiency by minimizing duplication of efforts among NGOs (i.e. through support to four NGO umbrella networks (which represent 90% of all NGOs) to promote knowledge sharing, collaboration and coordination, as well as completion of a study to promote areas of cooperation between NGOs and the private sector) as well as indirectly reaching additional beneficiaries through NGO sector development activities (i.e. through development of the Code of Conduct). In addition, the NDC "leveraged" the project funds by further raising $17.2 million (compared to a targeted amount of US$2.0 million).

However, there was a brief period in which project support was informally suspended. This disruption in NGO support led to some instances of NGOs accumulating debt in order to continue service delivery, NGOs facing legal charges for not honoring contracts, loss of investments made with project funds, lost opportunities for seasonal work, or inability to pay staff wages.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Moderately Satisfactory. Relevance of objectives and design is rated substantial. Achievement of the objective to provide social services to those who are poor, vulnerable or affected by the deteriorating socioeconomic conditions is rated substantial. Efficiency in the use of project resources is rated modest due to insufficient information.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

Institutional development in the sector, established through the NGO Development Center, is likely to be sustained. A fourth Palestine NGO project has been approved; however, only US$2.0 million has been allocated due to limited resources available under the Trust Fund for Gaza and the West Bank. Although the project will continue support for the NDC, the overall reduction in funds along with the volatility in the security, political, and socio-economic environment raises high risks for sustaining outcomes of this project.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
The project was a follow-up operation to the Palestine NGO I (FY98-03, US$15.1 million) and NGO II (FY01-06, US$25.4 million) projects. The project design drew upon experience from the Bank's long engagement with the NGO sector, and thus aimed to further develop the NGO sector through establishing a national-level focal point for NGO support. Implementing capacity was already largely in place through the preceding Bank projects, with additional assessments of NGOs conducted by the NGO Development Center and the Bank. The risk assessment was candid and realistic, with effective mitigation measures identified. The M&E framework was adequate, although longer-term development outcomes were not included given the crisis situation in the country.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:
The Bank team maintained an active role in the project despite challenging circumstances, including security conflicts, economic crises, and changes in Government leadership. The ICR notes (page 18) the importance of the Bank's role in maintaining effective coordination with other partners and donors. In August 2008, due to the escalating conflict, the Bank put projects in Gaza informally "on hold" until a number of conditions were met. The November 2009 Mid-Term Review concluded that the project was still on track to meet its development objectives and recommended continuation of the project, which resumed in February 2009 after all conditions were met. According to the ICR (page 18), "it is widely acknowledged among the PA [Palestinian Authority] and sectoral stakeholders that the Bank's support during project implementation made a significant positive difference in setting the stage for the future development of the NGO sector."

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Although Government support was strong during the preparation period, it was inconsistent over the remaining project period. In particular, the Palestinian Authority disengaged from the project for a period of time (from August 2008 to January 2009) because of political tensions with Hamas. The project environment was subsequently affected by tensions caused by the confiscation of assets of several NGOs in Gaza.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
The NGO Development Center (NDC), as the primary implementing agency, operated in a highly challenging environment. In the West Bank, barrier construction routinely disrupted social service work, there were difficulties in obtaining permits and visas for international consultants to enter Gaza, local NGO staff had difficulty leaving Gaza for training activities. There were also extreme difficulties in accessing goods needed for the implementation of sub-projects, as well as in enforcing compliance with legal obligations. Due to the border closures in Gaza, the main NDC staff (based in West Bank) were not able to directly supervise activities; however, the NDC's Gaza-based staff were able to assist through the use of video conferencing. However, the ICR (page 7) reports that the NDC maintained a high level of flexibility to cope with the environment, for instance revising budgets to ensure continuous technical support, adjusting work schedules according to power availability, and sharing field visits with other staff in light of fuel shortages. The NDC adequately carried out the required fiduciary, legal and safeguard activities for the project, even as it took on additional responsibilities in managing grants from other donors, and was "instrumental in bringing the project back on track following the suspension of activities.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The M&E framework contained realistic and measurable indicators. The M&E design included upgrading the existing information system under PNGO II to better report on outcomes, such as improved access to NGO services, effective targeting, and increased accountability. The participatory dimension of the M&E design was intended to encourage NGOs to evaluate NDC's support, as well as assist communities in monitoring and evaluating services delivered by the NGOs. Evaluative activities also included a beneficiary impact assessment.

b. M&E Implementation:
The upgraded, web-based Project Management Information System (PMIS) was developed and was capable of tracking and measuring progress on project indicators. An international M&E specialist worked closely with project staff to operationalize the PMIS and provide training on use of the system. The specialist also worked with the financial management staff to integrate the PMIS within the financial system, including adding six indicators to monitor grantees' progress in implementing work plans. Reporting formats were developed for each type of grant to report progress in the areas of finance, procurement, quality of services, and targeting. Community scorecards - both for NGOs to assess NDC, and for beneficiaries to assess NGOs - were also developed to assess sub-project performance, to be implemented in a minimum of three sessions (baseline, mid-term, final). A Beneficiary Impact Assessment (covering 35 out of 134 projects) was conducted by the Government in May 2010, and a Beneficiary Survey was conducted by the ICR mission in October 2011 (13 out of 50 NGOs), although a more detailed description of the methodologies would have been helpful.

a. M&E Utilization:
According to the ICR (page 8), the PMIS was used to track the use of funds (including effectiveness in targeting priority populations) and to allow sector specialists to verify accuracy of information.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:
The project was classified as a Category "B" project, triggering OP 4.01 on Environmental Assessment. Potential environmental impacts (such as water quality, wastewater treatment, air quality, natural resources management, solid waste and medical waste) were identified in the PAD based on experience with PNGO I and II and other demand-driven projects. All sub-projects were screened, and no substantial negative impacts were observed. An Environmental Management Plan (EMP) was in place for the project overall, with specific EMPs also developed for two sub-projects.

b. Fiduciary Compliance:
Financial management: Financial management performance was satisfactory overall, in terms of human resources, computerization of the network, and clear definition of responsibilities among the different departments. There were some initial problems with the accounting software, which had been specifically designed for the project, but were subsequently resolved. Annual audits were conducted and confirmed the adequacy of the financial management performance.


Procurement: Procurement procedures were clearly defined in the project manual, given that the responsible agency differed according to the type of grant. The NGO Development Center provided continuous and on-demand training and supervision to NGOs on procurement. Simplified procurement procedures were introduced in Gaza to take into account constraints due to the economic crisis. According to the project team, procurement performance was overall satisfactory with no cases of misprocurement.

Implementation of some sub-projects experienced delays due to restrictions on import of construction materials to Gaza, the absence of necessary equipment in the local market, delays in importing equipment from the West Bank or Israel. According to the ICR (page 6), it became difficult for suppliers to bid for various types of goods needed for the implementation of the NGO sub-projects because of the lack of control over variables such as time of delivery, price fluctuations, finding goods with right specifications in the local market. Additionally, "it was difficult for the NDC to push suppliers to abide by the legal obligations or to apply penalties specified in the contracts. Gaza was therefore forced to increasingly rely on goods that entered Gaza through informal channels through underground tunnels."

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
Efficiency in the use of project resources is rated modest due to insufficient information. 
Risk to Development Outcome:
High
High
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Moderately Satisfactory
Government performance is Moderately Unsatisfactory due to disengagement during part of the implementation period and confiscation of NGO assets. 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Lessons drawn by IEG:
  • Sustainability of NGO services remains an issue, particularly as the follow-up Bank operation will provide substantially less financing and the political and security situation continues to be highly volatile.
  • Continuous Bank engagement with the NGO sector through multiple projects has helped to sustain capacity strengthening efforts by building upon steps taken by previous projects.

Lessons from ICR (page 20):
  • The project's targeted approach was effective in providing "often the only means for reaching marginalized population groups and marginalized geographical areas without duplication of efforts of other organizations."

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is overall satisfactory in adhering to ICR guidelines. However, more information on specific project outputs (i.e. types and quantity of social services provided) is needed to clarify the results chain and verify project outcomes.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions