Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Social Fund II (SFII)


  
1. Project Data:   
ICR Review Date Posted:
03/14/2006   
PROJ ID:
P050601
Appraisal
Actual
Project Name:
Social Fund II (SFII)
Project Costs(US $M)
 27.7  36.3
Country:
Cambodia
Loan/Credit (US $M)
 25.0  33.8
Sector, Major Sect.:
Irrigation and drainage, Primary education, Other social services, Roads and highways, Water supply,
Agriculture fishing and forestry; Education; Health and other social services; Transportation; Water sanitation and flood protection
Cofinancing (US $M)
 TBD  2.2
L/C Number:
C3179      
   
Board Approval (FY)
  99
Partners involved
American Assistance for Cambodia (AAfC), OPEC 
Closing Date
12/31/2002 03/31/2005
         
Evaluator: Panel Reviewer: Division Manager: Division:  
Anju Vajja
Christopher D. Gerrard Alain A. Barbu IEGSG

2. Project Objectives and Components:

a. Objectives
The project objectives were:

(1) Provision of small scale, community-based sub-projects in the areas of social and economic infrastructure with an emphasis on the poor;
(2) Creation of short-term employment opportunities to absorb the increase in unemployment associated with post-crisis reverse migration of workers from Thailand, and with the demobilization of Cambodian army units;
(3) Strengthening of communities' capacity to implement development projects and sustain them;
(4) Improvement of donor coordination and co-financing;
(5) Improvement of poverty and district targeting.

In May 2001, a year before the expected closing of the SFII project, the Bank approved a Flood Rehabilitation Supplemental Credit (for US$10 million equivalent) which provided additional resources to the Social Fund of the Kingdom of Cambodia (SFKC) to address emergency needs in rebuilding small-scale infrastructure damaged in heavy flooding during 1999-2000. There was no change to the SFII credit agreement, given the similar objectives of both credits, nor any changes in the original components since those of the supplemental credit mirrored those of the original project.

b. Components (or Key Conditions in the case of Adjustment Loans):
The project had three components:

(a) Sub-project grants ($24.4 million at appraisal, $31.7 million actual) -- financed sub-grants for small-scale infrastructure and services projects proposed by communities, local groups, NGOs, and other groups. Eligible sub-project types included irrigation systems, bridges and culverts, water supply and sanitation systems, schools, vocational training centers and training, and health centers.
(b) Institutional support ($3.0 million at appraisal, $4.6 million actual) -- supported the costs of staffing, equipping, operating, training, and assessing the Social Fund.
(c) Unallocated: ($0.31 million at appraisal; $0.0 million actual) -- approximately 1 percent of the credit was left unallocated at appraisal to provide flexibility during implementation.

c. Comments on Project Cost, Financing, Borrower Contribution, and Dates
SFII was an immediate follow-on from the Social Fund I (SFI) project, and channelled funds through the same agency, the SFKC, as the first project.


IDA financed 91 percent of project costs and the Borrower less than 1 percent. AAfC provided $2.2 million of direct co-financing for schools and OPEC provided an additional $7.8 million in a separate OPEC project which started in March 2001 and closed in December 2005. Ten NGOs contributed an additional $4 million at the sub-project level.

The closing date of the project was extended by 15 months due to implementation delays at the beginning of the project, and by an additional year due to the approval of the supplemental flood rehabilitation credit. Initial implementation was delayed due to political uncertainties and due to the introduction of new procedures, so that it was not until May 2000 -- ten months after project effectiveness -- that operations again reached the level achieved during SFI.


3. Relevance of Objectives & Design:

Overall relevance is rated substantial. The project is consistent with both the government and CAS objective of rural development. Allocating at least two-thirds of total sub-project financing rural economic infrastructure -- irrigation, rural roads, water supply (PAD, p. 3) -- is justifiable given other donor programs' focus on education and health. Given the sustainability concerns regarding water supply sub-projects in SFI, SFII established a Sustainability Program Unit (SPU) and hired a sustainability advisor.

The current government and Bank Country Assistance Strategy emphasizes the importance of strengthening the capacity of government institutions, yet the project did not address this objective. The project operated solely through an autonomous agency, SFKC.

4. Achievement of Objectives (Efficacy) :

Overall efficacy is rated modest.

Objective (1) is rated substantial. The project financed 1,416 small- to medium-sized economic and social infrastructure sub-projects throughout the country, which benefited over 5 million people according to SFKC estimates. According to the Independent Technical and Beneficiary Assessment (ITBA) completed in late 2004, benefits included increased school enrollment, particularly among girls; safer school environments; improved access to schools and markets; decreased impacts from flooding; enhanced trade in agriculture and other local products; and added income from employment opportunities. However, school construction was to be limited to less than one-third of total sub-project approvals in value terms, since other donors' programs focused on education and health, and at least two-thirds was to be allocated towards rural economic infrastructure (PAD, p. 3). Instead, over 50% was allocated to schools and only 45% to rural economic infrastructure.

Objective (2) is rated modest. The project did not include special incentives or provisions for hiring returning workers or former military personnel, and the backgrounds of employees hired were not tracked. (ICR, p. 4).

Objective (3) is rated modest. The ICR concludes that "the project appears to have had a positive impact" on community capacity to implement and sustain sub-projects, as evidenced by (a) high levels of ownership/community contribution (time, in kind, labor and money); (b) participation by 60% in other community development activities; and (c) use of skills gained through SFKC experience in other development activities by 80 percent (p. 4). This conclusion is based primarily on the results of the ITBA, which interviewed over 1000 individuals, including in-depth interviews with Project Support Committee (PSC) members, local and national authorities, and over 500 randomly selected beneficiaries. However, it not clear to what extent capacity enhancement can be attributed to the project without baseline surveys or with and without project comparisons. For instance, that 60 percent of the members indicated that they went on to participate in other development projects does not demonstrate capacity enhancement unless one can specify how many of these had never before participated in development projects prior to SFKC.

Objective (4) is rated modest. The project was able to improve the level of co-financing, but there is no indication in the ICR on improvements in donor coordination insofar as strategic consensus, procedures, and policy are concerned.

Objective (5) is rated modest. The project proposed to improve poverty targeting, based on new poverty data (PAD, p. 4). However, the poverty targeting objective was not met fully, according to the ICR, because of insufficient socioeconomic data for certain areas of the country and the lack of a coherent targeting mechanism. The SFKC used a set of different and competing prioritization tools (ICR, p. 5).

5. Efficiency:

There is insufficient information to rate efficiency.

No economic rate of return (ERR) was calculated at appraisal or at closure because "standard rate of return analysis is not readily applicable to a large number of small, demand-driven, and disparate sub-projects." (ICR, p. 6) However, while the ERR could not be calculated ex-ante because the sub-project investments are not known at appraisal, a sample of ERRs could should have been attempted for typical sub-projects supported by the project. The PAD had also indicated (p. 9) that relatively simple ERR calculations would be calculated for larger economic infrastructure sub-projects.

The ICR indicates that operational costs were contained to the agreed 8% of total project costs, lower than other public agencies and below the average of social funds included in a recent Bank comparative study (10-13%). The Bank's 2003 comparative study indicated that social funds’ overhead expenses were similar among countries, falling within a range of 7–13 percent of total program costs (Source: Evaluating Social Funds: A Cross-Country Analysis of Community Investments). Thus the SFII was in the lower part of this range.

6. M&E Design, Implementation, & Utilization:

M&E design was weak. There was little emphasis on systematically measuring outcomes and impacts as evidenced by the lack of outcome and impact indicators, targets for key indicators, baseline surveys, and follow ups. The ICR notes that "it should be acknowledged that at the time the SFII was prepared, the Bank in general was placing much less emphasis on indicators and mechanisms for monitoring and evaluation, compared to the strong focus the Bank now has on measuring outcomes and impact (p. 9)." However, even the few indicators that were mentioned in the PAD were monitored and tracked only sporadically (p. 9).


SFKC did not widely disseminate the lessons learned from its experience in local development, community capacity-building, governance, and partnerships until the end of the project (ICR, p. 7).

7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative):

According to the PAD, SFKC only rebuilds or rehabilitates existing structures and each includes an environmental assessment. Further, if an assessment revealed substantial environmental impact, appraisal was to be halted and further impact studies were to be conducted. However, no study was carried out on the cumulative impact of sub-projects, such as the cumulative impact on the water table of rehabilitating/rebuilding over 5000 water supply systems.


SFII created an Internal Audit Unit, and adopted local shopping procedures for sub-projects valued below US$250,000, etc. in response to circumstantial evidence of procurement irregularities in mid-2000 (p. 7). The ICR does not report on community capacity to undertake fiduciary management responsibility.


8. Ratings:
ICR
ICR Review
Reason for Disagreement/Comments
Outcome: 
SatisfactoryModerately Satisfactory[The ICR's current 4-point scale does not allow for a "moderately sat." rating]. Project achieved most of its major relevant objectives but with significant
shortcomings.
Institutional Dev.: 
ModestModest
Sustainability: 
LikelyNon-evaluableThe evidence provided in the ICR for sustainability of sub-projects is mixed and insufficient for a "likely" rating.
Bank Perf.: 
SatisfactorySatisfactoryAlbeit marginally so, due to:
(a) Lack of exit strategy for SFKC given the focus of the government strategy and CAS on building government capacity.
(b) Weak poverty targeting strategy, monitoring and evaluation design and dissemination of lessons learned.
Borrower Perf.: 
SatisfactorySatisfactoryBut the implementing agency did not track all the M&E indicators specified during implementation.
Quality of ICR: 
Satisfactory

NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness.

9. Lessons:

The ICR has extracted a number of relevant lessons from the experience with this project.
  1. Monitor outcomes and evaluate impact to assess development effectiveness. To assess development effectiveness input and output indicators (amount disbursed, sub-projects completed) should be supplemented by outcome and impact (sustainability, benefits) indicators. For projects benefiting a large proportion of the population, use of household surveys to monitor impact (for example capacity enhancement) should also be explored.
  2. Find mechanisms to ensure appropriate targeting. Projects aimed at supporting poor communities and specific beneficiary groups should have a coherent targeting mechanism in place. In addition, indicators that measure whether project benefits are reaching the target groups should be developed and monitored to facilitate adjustments during implementation.
  3. Tackle transparency issues systematically especially for an effective procurement process. Well-defined procedures, adequate training, rigorous internal monitoring, regular independent external evaluations, and appropriate and enforceable penalties and a commitment to invoke them is required as well as continuity of Bank staff, frequent site visits, and through country knowledge.
  4. Communicate through regular and consistent dialogue with all stakeholders to achieve sustained impact. For example, SFKC could have a substantial institutional impact on the capacity of government agencies had it communicated and collaborated more extensively with them through dialogue, data sharing, technical assistance and training sessions.
  5. Plan for hand-over or exit strategy to ensure lasting institutional development impact. For all Social Funds and especially for this one where the implementing agency enjoys a high degree of autonomy, it is important to include mechanisms to ensure that the valuable processes, skills, and systems which are developed can be adopted by and/or passed on to national and local institutions.

10. Assessment Recommended?  Yes

          Why?  Post-conflict countries often face the urgent need of restoring basic services and infrastructure. Given low government capacity, social fund projects address these needs through autonomous agencies, rather than build longer-term institutional capacity by working with the weak public entities/bodies. However, in many cases this has resulted in unsustainable infrastructure. It would be important to assess if by-passing the government provides a more effective and sustainable means of providing basic infrastructure and services than directly building up the government's institutional capacity, especially in those post-conflict countries where the Bank is increasing its aid.

11. Comments on Quality of ICR:

The ICR is rated satisfactory overall, but it lacks: (a) Clear and systematic presentation of tables (for example, the number of different types of sub-projects, and the average unit cost of each type of sub-project on page 16).
(b) ERRs for larger economic infrastructure sub-projects (as indicated in the PAD).
(c) Beneficiary assessment questionnaire/analysis, which is typically an annex in ICRs for projects in which beneficiary assessments have been conducted.

(ES-Rev4B-Dec/05)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions