Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Second Magdalena Medio Regional Development Project


  
1. Project Data:   
ES Date Posted:
02/25/2005   
PROJ ID:
P057692
Appraisal
Actual
Project Name:
Second Magdalena Medio Regional Development Project
Project Costs(US $M)
 6.35  6.27
Country:
Colombia
Loan/Credit (US $M)
 5.0  4.92
Sector, Major Sect.:
General agriculture fishing and forestry sector, Central government administration, Other social services, Agro-industry,
Agriculture fishing and forestry; Law and justice and public administration; Health and other social services; Industry and trade
Cofinancing (US $M)
 --  --
L/C Number:
L7078      
   
Board Approval (FY)
  2
Partners involved
-- 
Closing Date
12/31/2003 05/28/2004
         
Prepared by: Reviewed by: Group Manager: Group:  
Soniya Carvalho
Nalini B. Kumar Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
The main objectives of this second LIL were to: test new forms of managing and operating the Magdalena Medio Regional Development Program (PDPMM) which result in: (a) an increase in the influence of citizens in local and regional development; (b) local institutions which are responsive and accountable to citizens; and (c) tangible social and economic benefits. In this regard the project was to focus on learning in four areas involving testing of: (i) new organizational and operational mechanisms for managing PDPMM, which increase the role of community organizations, take into account the diverse conditions in the region particularly the armed conflict and gradually decrease dependence on the Corporation for Development & Peace Magdalena Medio (CDPMM); (ii) approaches for strengthening local and regional institutions (private & public) so that they respond better to citizens' needs and priorities, and are accountable for their actions; (iii) approaches for implementing productive projects which increase the probability of their success and contribution to the economic and social development of the region; (iv) approaches to developing and implementing social initiatives that would respond to the needs of vulnerable communities (in extreme conditions of poverty and/or impact of armed conflict) and eventually allow them to pursue integrated and sustainable development initiatives. The PDPMM is a government program initiated in 1995 with support from the National Petroleum Company and the Catholic Church for promoting regional development and peace in conflict-affected areas. According to the PAD, CDPMM is a change agent which enjoys high credibility among all stakeholders.

b. Components
The project comprised 4 components:

(1) Program leadership and increased role of communities (total project cost, appraisal estimate: $3.41m; actual: $3.32m)
(2) Strengthening of local and regional institutions (total project cost, appraisal estimate: $0.89 m; actual: $0.95m)
(3) Productive subprojects (total project cost, appraisal estimate: $1.75m; actual: $1.87m)
(4) Social subprojects in vulnerable communities (total project cost, appraisal estimate: $0.30m; actual: $0.13 m)

c. Comments on Project Cost, Financing and Dates
This was a second LIL for an estimated total project cost of $6.35m. The actual total project cost was $6.27m. Bank financing was estimated at $5m. The actual Bank financing was $4.92m. The project closed about 5 months after the original closing date. The ICR does not provide a reason for the delay. According to Regional comments, the government requested the extension of the closing date to complete two studies.


3. Achievement of Relevant Objectives:

The project objectives appear to have been achieved based on limited evidence in the ICR. According to Regional comments, in the beginning of the project, the semi-annual M&E reports provided more narrative, impressionistic and qualitative information than harder evidence of tangible benefits. Over time, the project staff was able to provide reports with a better balance between qualitative and quantitative data so that, by the beginning of 2003, the project generated benchmarks that were then used as a baseline to better assess the results....the annual implementation plans, the semi-annual M&E reports, the external evaluation, the study on best practices of 4 sub-regional projects of Component 3 and other technical reports, detail the individual, organizational, institutional learning processes and behavioral changes that came about as a result of the project". In any case, given that this was the second LIL and had significant learning objectives, greater attention was needed to establishing systematic M&E especially of outcomes and impact early on. The ICR points out that "the monitoring system was developed at a relatively advanced stage of the project.... its usefulness was reduced due to this time lag" (ICR page 20-21). The proposed project performance assessment will verify the extent to which the LIL actually contributed to the testing of approaches and informed both decisions to replicate and scale-up and the design of follow-on operations.

4. Significant Outcomes/Impacts:

The government adopted, in the National Development Plan 2002-2006, the strategy and approach of regional peace and development programs inspired by the two LILs.
  • The management of project resources by CDPMM under this project resulted in its managing the significantly larger Peace Laboratory Project financed by grant resources from the European Union in the amount of EUR34m.
  • The project leveraged $7.5 from other sources for every dollar provided by the Bank (ICR page 7).
  • The Director and other senior project staff are frequently called to act as facilitators in the region and their advice is sought after by government authorities and international and national organizations.
  • The project contributed to the development of the Participatory Regional Planning System which takes the region as a unit, and, through a series of participatory planning activities, influences the development plans in each of the municipalities.
  • The palm project supported under the productive subprojects component leveraged additional resources and delivered products to market at an acceptable cost/yield ratio.
  • According to Regional comments, the Presidential Decree of February 7/05 on attention and prevention of vulnerability and displacement adopted the core concepts, methods, and approaches promoted by the project.

5. Significant Shortcomings (include non-compliance with safeguard features):

1. A QAG quality at entry assessment pointed out that the project could have been improved by building more explicitly on an analysis of socio-political aspects (beyond the civil conflicts) and incorporating such considerations into project design and implementation arrangements. The assessment also highlighted the need for greater specificity about the processes and activities summarized under "learning" and the way they are designed. Given the project's focus on testing approaches for capacity building and institutional and organizational change, the assessment pointed to the need to bring in expertise on institutional and organizational development in preparing the project. According to the ICR, safeguards were not addressed in the design of the project (ICR page 15). According to Regional comments, the quality at entry issues were addressed during implementation. 2. A supervision mission in May 2003 found some weaknesses in how the project dealt with environmental concerns, particularly regarding integrated pest management, and the ICR rates achievement of the environmental objectives/outputs of components as "modest" (ICR page 33). According to Regional comments, corrective actions were taken to address the weaknesses found during supervision. But the Regional comments also note that the environmental evaluation found that "there were problems with storage and manipulation of pesticides that may be contrary to Bank guidelines"--a non-trivial issue in OED's opinion which will be verified through the project performance assessment.
3. A final evaluation of the project expressed concern that the strength of project-supported entities might have come to some degree at the expense of established local organizations such as the Community Action Boards (ICR page 12). Furthermore, the government noted that a limiting aspect of the project was the absence of a project coordination unit at the national level. "Within the National Technical Committee, the National Planning Department offered some guidance or coaching. To a lesser degree, it provided technical and administrative support for the committee. The project would have been better served by having a basic nucleus of professionals dedicated specifically to the CDPMM" (ICR page 20).

6. Ratings:ICROED ReviewReason for Disagreement/Comments
Outcome:
SatisfactorySatisfactoryGiven that this was the second LIL and had significant learning objectives, greater attention was needed to establishing systematic M&E especially of outcomes and impact early on.
The ICR points out that "the monitoring system was developed at a relatively advanced stage of the project.... its usefulness was reduced due to this time lag" (ICR page 20-21). Regional comments note that the M&E system though late in start-up was adequate and useful. The proposed project performance assessment will verify the extent to which the LIL actually contributed to the testing of approaches and informed both decisions to replicate and scale-up and the design of follow-on operations. Furthermore, Regional comments note that the environmental evaluation found that "there were problems with storage and manipulation of pesticides that may be contrary to Bank guidelines"--an issue that will also be verified through the project performance assessment.
Institutional Dev.:
HighSubstantialA final evaluation of the project expressed concern that the strength of project-supported entities might have come to some degree at the expense of established local organizations such as Community Actions Boards (ICR page 12). Furthermore, the government noted that a limiting aspect of the project was the absence of a project coordination unit at the national level. "Within the National Technical Committee, the National Planning Department offered some guidance or coaching. To a lesser degree, it provided technical and administrative support for the committee. The project would have been better served by having a basic nucleus of professionals dedicated specifically to the CDPMM" (ICR page 20). These issues need further examination before the project canh be rated "High" on Institutional Development Impact.
Sustainability:
Highly LikelyLikelyThe ICR does not provide evidence to back-up a highly likely sustainability rating. While funding does not seem to be a problem, the sustainability of project benefits needs to be watched: "local residents continued to be displaced as a result of the conflict, which, in turn, eroded hard-earned project achievements (ICR page 12).
Bank Performance:
SatisfactorySatisfactoryOED concerns about the adequacy and usefulness of the project's M&E system in helping to test approaches and inform both decisions to replicate and scale-up and the design of follow-on operations will be verified through the project performance assessment. The project performance assessment will also examine concerns on environmental aspects.
Borrower Perf.:
SatisfactorySatisfactory
Quality of ICR:
Unsatisfactory

7. Lessons of Broad Applicablity:

Based on the ICR, this Review highlights the following lessons:
1. M&E is critical, especially in LILs and second LILs. Testing cannot yield its full benefits unless impacts are systematically evaluated and implications carefully drawn.
2. Community building cannot be rushed. Process, institutional, and behavioral change take time to materialize and may need support over the long haul.
3. Productive activities are likely to fail unless markets for the outputs are assured. Just supporting front-end inputs is likely to be inadequate. Sound market analysis is critical.
4. Systematic attention to safeguard issues is critical in design, implementation, and M&E.
5. Institutional impacts should be carefully monitored to avoid negative effects. Process, institutional, and behavioral objectives need systematic M&E (including through qualitative techniques). M&E for such objectives needs to be carefully designed and implemented to determine pros and cons of various approaches and make mid-course changes as necessary.

8. Audit Recommended?  Yes

          Why?  To verity the ratings and distill lessons relating to ways of bringing about institutional and behavioral change in conflict-ridden communities.

9. Comments on Quality of ICR:

The quality of the ICR is unsatisfactory. The ICR should have systematically presented quantitative and qualitative data to demonstrate achievement of the project's testing and learning goals, and discussed systematically the pros and cons of each approach tried under the project. As currently presented, the ICR contains several assertions with limited evidence to back them up. The ICR does not report adequately on safeguard compliance. It does not sufficiently examine the possible institutional harm the project might have caused, simply reporting instead that "A final evaluation of the project expressed concern that the strength of project-supported entities might have come to some degree at the expense of established local organizations such as the Community Action Boards (ICR page 12). The ICR does not explain the underlying methodology for the economic analysis presented in Annex 1: "Economic analysis of 5 subprojects..show satisfactory IRRs. In 3 out of 5 IRRs are greater than 50%". The ICR does not provide a reason for the (about 5 month) delay in project closing compared with the original closing date.

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions