Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Emergency Community Empowerment And Public Works Project


  
1. Project Data:   
ICR Review Date Posted:
10/21/2005   
PROJ ID:
P077533
Appraisal
Actual
Project Name:
Emergency Community Empowerment And Public Works Project
Project Costs(US $M)
 42.00  47.16
Country:
Afghanistan
Loan/Credit (US $M)
 42.00  47.16
Sector, Major Sect.:
Irrigation and drainage, Central government administration, Other social services, Roads and highways,
Agriculture fishing and forestry; Law and justice and public administration; Health and other social services; Transportation
Cofinancing (US $M)
 0  0
L/C Number:
CH009      
   
Board Approval (FY)
  02
Partners involved
EC, UK, USA, Canada, Germany, Denmark, Japan, (all parallel i.e. excluded from project cost). 
Closing Date
12/31/2004 12/31/2004
         
Evaluator: Panel Reviewer: Group Manager: Group:  
Ridley Nelson
Kris Hallberg Alain A. Barbu OEDSG

2. Project Objectives and Components:

a. Objectives
The objectives, which responded to a country context requiring urgent action, were related to public works and community empowerment. With respect to public works, the objectives were to provide employment in rural areas to as many people and in as short a time as may be feasible and to rehabilitate, in a sustainable way, irrigation capacity and provincial and district level roads. With respect to community empowerment , the objective was to support participatory planning and strengthening of local governments, to help villages rebuild their capital base through grants and empowerment, and to provide a mechanism for linking community development needs with line agency and NGO service providers.

b. Components (or Key Conditions in the case of Adjustment Loans):
Corresponding to the two categories of objectives, the components were as follows (costs as at appraisal):

Public Works: (i) District Grants for Sub-Projects (US$18.4 million); (ii) Technical Assistance for Implementation (US$1.6 million); (iii) Monitoring and Preparation of a Longer-term Program Proposal (US$0.5 million); and, (iv) Incremental Operating Expenses (US$0.5 million). Community Empowerment: (i) Village Grants program (US$16.0 million); (ii) Implementation Support (US$4.8 million); and, (iii) Monitoring and Evaluation Studies (US$1.2 million).

c. Comments on Project Cost, Financing, Borrower Contribution, and Dates
This was classified as an emergency project and was 100% financed by IDA i.e. with no borrower contribution. During implementation there were substantial changes in the allocation of project costs. This was mainly due to the urgency of rehabilitation for the Salang Tunnel and to expand consultant services in order to scale up the community empowerment with supplemental projects funded by both the Bank and other donors. Actual expenditure for public works was about two-thirds of the appraisal estimate, for community empowerment was a little over one third of appraisal, and for consulting services and training was nearly four times appraisal. These changes were partly due to the substantial scaling up of the whole program. The project closed on schedule after 2 1/2 years and was followed by further projects doing similar work.


3. Relevance of Objectives & Design:

Overall the project was relevant. The objectives were highly relevant to the strategy and situation of Afghanistan both at the time and now. Design was satisfactory but with a few weaker areas including over-optimism with respect to institutional capacity as noted by QAG, insufficient attention to M&E design given that it was a pilot exercise.

4. Achievement of Objectives (Efficacy) :

There were no outcome indicators set at appraisal beyond some projections of component expenditure (an input measure), although some targets seem to have been set and amended early in the program. Given the rapidly changing situation following approval, with a greatly expanded program using parallel donor financing, it is very difficult to attribute all outcomes to the IDA-funded project. The targets set for the expanded program were: 4,500 elected Community Development Councils, 4,500 villages with community development plans, 3,000 sub-projects, US$72 million in sub-project grants, 9,000 community members trained, 3,000 facilitation partners trained and 120 staff trained. However, original targets were much less, for example 600 to 800 supported villages. In the following, the investments are directly attributable to the project, but the support and empowerment only more loosely attributable. However, it should be noted that the project can be partly attributed with triggering substantial and rapid scaling-up. The objective of providing employment, was fully achieved creating 3.9 million labor days, approximately what was projected. The objective of rehabilitating infrastructure was achieved, repairing over 4000 km of rural roads, repairing 10 irrigation systems, 16 schools and 40 water storage facilities. The objective of supporting participatory planning and strengthening local government was probably largely achieved, although data on performance in this area is limited in the ICR. There have been a number of operational procedures established. Regional staff have pointed out that an Operational Manual was developed in 2003 through a series of consultations giving buy-in from key stakeholders which has since been updated drawing on further consultation and implementation experience. Over 6,000 community development plans were prepared across 101 Districts which the ICR suggests improved local governance, however, the ICR also notes that geographical targeting and equity presented problems. Villages with elected Community Development Councils now number over 6000. The training of field staff and villagers also contributed to this objective, although the quality and outcome of that training does not yet appear to have been measured. The objective of rebuilding the capital base of villages was probably largely achieved, about 6,000 sub-projects were completed over two years, however this is an input and completion of structures does not necessarily imply impact. The ICR was not able to show what impact this rebuilding had, for example on transport costs, land values, or agricultural productivity, nor provide evidence on quality, due presumably to limited data. However, it was still early to assess impact. The objective of providing a mechanism to link community development needs with line agencies and NGOs is said to have been largely achieved, although there remain weaknesses in the capacity of line agencies.

5. Efficiency:

No economic analysis was attempted. However, the ICR presents some evidence of the costs of service delivery notwithstanding some definitional problems. The ICR notes that it cost $1 to deliver $1 of block grant to communities in the first year when the system was being established but that this fell to 26% in the second year. While this still appears a high cost and could no doubt be reduced further, it is not outside the range of NGO operations reviewed by OED in geographically similar and, in fact, less conflict-ridden environments.
6. M&E Design, Implementation, & Utilization:

As is openly stated in the ICR, M&E was weak partly due to lack of a baseline survey and partly due to lack of skills. The ICR rates M&E as unsatisfactory. For a pilot project, and a large one, this is a significant weakness.
7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative):

No safeguard or unintended impacts were noted in the ICR. There is not much evidence of environmental attention during the planning stage in infrastructure components, although there do appear to have been some guidelines. With an average size of $10,000 per grant in widely scattered locations and much of it in the nature of repair, significant widespread environmental impact would be unlikely and it would have been unreasonable to expect any sophisticated environmental assessments. Also, the appraisal report indicates that the project would seek assurances that participating communities do not engage in poppy cultivation but project performance in this direction was not addressed in the ICR.

8. Ratings:
ICR
OED Review
Reason for Disagreement/Comments
Outcome: 
SatisfactorySatisfactory
Institutional Dev.: 
SubstantialSubstantial
Sustainability: 
LikelyNon-evaluableThere remain considerable uncertainties given the early development stage of the interventions. The ICR provides insufficient evidence for a "Likely" rating at this point. While, on the positive side, the ICR points to broad community commitment, on the negative side, there are concerns about infrastructure maintenance, notwithstanding community maintenance agreements, particularly given the current very low rural income environment. The ICR acknowledges that "...the sustainability of the economic benefits of the infrastructure investments remains doubtful" (ICR page 7), although staff note that this refers primarily to the tertiary roads. Much of the maintenance challenge on new structures lies in the future.
Bank Perf.: 
SatisfactorySatisfactoryBut M&E was very weak despite this being conceptualized as a large scale pilot exercise as noted in the ICR (ICR page 5).
Borrower Perf.: 
SatisfactorySatisfactory
Quality of ICR: 
Satisfactory

NOTES:
- When insufficient information is provided by the Bank for OED to arrive at a clear rating, OED will downgrade the relevant ratings as warranted beginning July 1, 2006.
- ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness.

9. Lessons:

The ICR offers well-crafted lessons. The following are summaries of the most important ones:

1. In a difficult post-conflict situation, performance-based contracting of service providers can be an effective delivery mechanism.
2. In a post-conflict situation, the understandable need for rapid results (the "peace dividend") should not over-ride the ultimately more important capacity building.
3. Maintaining engineering quality sufficient to ensure sustainability and performance is important notwithstanding the need for urgency.
4. Public works programs can be a very blunt instrument as a safety net. Expectations about the ability of these programs to target the poor and vulnerable should be realistic. Self-targeting through wage levels needs seasonal wage knowledge and careful wage setting.
5. The process of involving women in a very conservative social environment needs careful nurturing to push strongly towards empowerment of women but without excessive perceived threat to cultural values.

Another lesson, with echoes in other similar experiences, is in the design of training for women aimed at income generation through self-employment. This is very challenging in such societies, as has been found in the Pakistan AKRSP. Given the relatively narrow range of product options, there needs to be particularly careful assessment of market potential for products and associated support in the form of micro-finance and follow-up.


10. Assessment Recommended?  Yes

          Why?  To verify the ratings, including institutional development impacts at both the local government and community levels (specifically how the project changed local dynamics). It would also be important to get an early reading of performance in a "new" country like Afghanistan and there appear to be important, and evolving lessons to be learned of wider applicability.

11. Comments on Quality of ICR:

This is a well-written ICR. It sets out the story with candor, noting weaknesses. Generally, it is easy to follow what the project did and what the problems were. The lessons are well formulated and clearly draw from the project experience. The data is provided to the extent possible even though M&E, as acknowledged by the ICR, was very weak. It offers a good summary of the other donor views (favorable) which is rare. However, even with weak data and the challenge of measuring impact so early, it could have somewhat better addressed qualitatively, if not quantitatively, outcome questions such as the transport cost impact of the improved roads, increase in agricultural production resulting from renovated irrigation, extent and nature of local government strengthening, degree of responsiveness of decision-making and service delivery to community needs, and impacts on existing leadership patterns resulting from the inclusion of new actors in Community Development Councils (ICR page 13).

(ES-Rev4-Jul/05)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions