Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Natural Protected Areas Project (GEF)


  
1. Project Data:   
ES Date Posted:
02/24/2004   
PROJ ID:
P052209
Appraisal
Actual
Project Name:
Natural Protected Areas Project (GEF)
Project Costs(US $M)
 32.8  58.8
Country:
Mexico
Loan/Credit (US $M)
 25.2 grant  23.5
Sector, Major Sect.:
General agriculture fishing and forestry sector, Central government administration, Sub-national government administration, Other social services,
Agriculture fishing and forestry; Law and justice and public administration; Law and justice and public administration; Health and other social services
Cofinancing (US $M)
 7.6  35.3
L/C Number:
     
   
Board Approval (FY)
  97
Partners involved
Government, NGOs 
Closing Date
06/30/1998 12/31/2002
         
Prepared by: Reviewed by: Group Manager: Group:  
Peter W. Whitford
Christopher D. Gerrard Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
NOTE: This Evaluation Summary, like the ICR, will focus on the restructured project of May 1997, as there is insufficient information on the original project of 1992 to make an informed judgement. Inclusion of the earlier period, however, would likely lower ratings in all areas.

The objectives of the restructured project (PD, para 26) were:

a) to protect unique biodiversity in eligible biosphere and special biosphere reserves;
b) to strengthen the management of protected areas at the reserve level;
c) to promote local participation, including indigenous communities, in the implementation, operating and management plans for protected areas; and
d) to ensure long-term recurrent cost financing for core protection and conservation activities.

b. Components
1. Reserve Conservation Program - protection, community outreach, and training activities in 10 protected areas, including Technical Advisory Councils and income generating activities.

2. Central Coordination Programs - support activities at the national level.
3. Permanent Endowment for Protected Areas - establishment and operation of a permanent endowment fund (FANP), the income from which would fund operating costs at the 10 reserves.

c. Comments on Project Cost, Financing and Dates
The estimates of project cost, financing and implementation period at restructuring appear to have been realistic. The 50% "cost overrun" reflects additional government and other donor contributions to the program, presumably for enhancements to the project scope, and is not unusual for projects of this kind. However, apart from a brief reference in 4.1, the ICR does not compare the additional inputs and outputs nor estimate cost-effectiveness.


3. Achievement of Relevant Objectives:

The ICR shows a "Revised Objective" in respect of a): "to implement protection/ conservation programs in ten biosphere reserves in high priority ecosystems containing endemic and/or endangered species of global importance," but incorrectly attributes this to the Restructured Project of 1997, the objectives of which are quoted above (objectives b), c) and d) were not changed). The change from "biodiversity protection" to "implementing conservation programs" was a significant narrowing of project scope and, according to information from the task team leader, dates from a workshop in February 1998, which also established the indicators for measuring progress on the conservation objective; it also corresponds more closely with the actual thrust of the restructuring appraisal and the implementation concerns of the Bank and the Recipient and also those of the GEF. The ICR reports that the following indicators were monitored to assess the outcome of objective a):

i) rate of habitat conversion;
ii) trends in observation of indicator species;
iii) area under sustainable use; and
iv) number of residents adopting sustainable use practices.
(These indicators are similar to those proposed in Annex 10 of the PD.)
Unfortunately, the ICR does not provide details on how these indicators (especially "sustainabile use practices") were defined or measured, though the task team leader has since provided some information, for example, on the use of satellite imagery for indicator i), the influence of factors beyond the control of the project (like floods and landslides), and the difficulty of projecting what would have happened without the project. However, the numerical results of monitoring to date are not presented in detail with respect to the 10 reserves. Section 4.2 of the ICR states that "The preliminary data show that habitat conversion rates have decreased in some areas...but have increased in others, with the highest rates (ca. 0.7% per year) occurring in Mariposa Monarca and Ria Laagartos. Deforestation rates since the beginning of the project also show mixed upward and downward trends, which could reflect the short time that the project has been in existence, or an increasing level of threat beyond what the project can address. One reserve, Sian Ka'an, actually shows an increase in vegetative area due to restoration projects." (The first reserve mentioned is the winter refuge of the well-known Monarch butterfly). Under either wording of objective a), this is a worrisome outcome, to which the ICR gives scant attention. It is appreciated that cause and effect relationships are complex and that some more time may be needed to establish the longer-term project outcomes but the ICR could well have devoted some space to analyzing thsi question.
An added concern is that the follow-up project (SINAP II) was begun before the ICR for the first project was prepared and may not have fully incorporated the implications of these findings. It is understood that a proposal from the Recipient to undertake the ICR before June 2001 was not adopted by the Bank. However, the task team maintains that the SINAP II appraisal did rely on "extensive and intensive analysis of SINAP I". According to GEF, SINAP II was based on lessons form the Mid-Term Review.
On the basis of monitoring results to date, as reported in the ICR, achievement of objective a) is therefore rated U, though the point is conceded that better results may be obtained over time from the protected area managements put in place by the project.
Regarding objective b), the project has established a core staff of competent and trained staff at each reserve and published management plans. Funding is now adequate. This objective is therefore rated S, approaching HS.
Little information is available regarding objective c), local participation. The formal target of establishing technical Advisory Councils, including indigenous people and other stakeholders was achieved at each reserve but it is not clear how effective they have been in addressing the social problems of encroachment, deforestation etc. The ICR notes that 60 community-based social development projects have been supported (presumably the "income generating activities" foreseen at appraisal), at a cost of more than $4 million, but there is no word on their results - either in social or economic terms. It is hard to see why no provision was made for pre- and post-project socioeconomic surveys, though "social assessments" have been made in six areas, to measure progress in achieving this objective and to gain insight into the social issues which are preventing or delaying the achievement of objective a), as well as to provide a sound basis for SINAP II. This objective is therefore rated as "non evaluable".
Achievement of objective d), establishment and operation of FANP, has exceeded expectations and is clearly HS.


4. Significant Outcomes/Impacts:

The ICR generally does a good job of describing and evaluating the project outcomes, especially in regard to national protected area management, management of reserves, and financing issues, though it remains rather light on biodiversity conservation and people/ parks issues. Among the highlights were:
  • substantially increased funding of protected area management
  • national regulations for protected areas published
  • elevated role of the national council for natural protected areas (CONANP)
  • permanent core staff at all 10 project reserves and training of all reserve staff
  • management plans published for all reserves
  • annual operating plans in use
  • Technical Advisory Councils established for all reserves, including indigenous people, which are participating in review of plans and implementation of social strategies
  • social strategies include indigenous peoples development plans
  • more than 60 community-based sustainable development projects supported
  • monitoring and evaluation system in place
  • six-year, coherent programs for protecting biodiversity developed through participatory means
  • independent external evaluations at mid-term and completion
  • FANP established and operating with financial results exceeding appraisal projections
  • FANP administration costs a reasonable 12%
  • FANP endowment not depleted and now at $41.5 million (including SINAP II)
  • project co-financing and other donor support of FANP greater than expected and GEF funds well leveraged
  • active Mexican NGO participation in the project
  • project successes have spilled over into other protected areas
  • SINAP II designed and launched based on success of this project

As the above outcomes indicate, institutional impacts have been High: a powerful, permanent funding mechanism has been established and permanent management structures set up for the reserves. However, CONANP continues to have problems. For the same reasons, project sustainability - a major challenge for GEF biodiversity projects - is rated Highly Likely, even if external donor support were to be phased out, demonstrating the value of the endowment approach.

5. Significant Shortcomings (include non-compliance with safeguard features):

Quality at Entry
The challenge in 1997 was to turn around a GEF project directed at important biodiversity goals but which had achieved little since its inception in 1992 (less than $4 million out of a total grant of $25 million disbursed), owing to a combination of institutional instability and economic crisis. The "big idea" was to introduce the endowment concept which had been successfully pioneered elsewhere (though the PD does not provide details). A major contribution of the Bank was to transfer knowledge on how to do this. The PD, therefore, is strongly focussed on the new financing mechanism and associated institutional provisions and transition arrangements. The biodiversity and social objectives, which were presumably carried over from the original project, got relatively little attention at appraisal. For example, there is no detailed discussion of the threats to biodiversity, or of endangered species or ecosystems, and each of the protected areas is described in a half page or less. Only the main principles of the government's 1996 protected area strategy are mentioned. Project benefits were described largely in terms of strengthened financial and institutional capacity. Identified risks were also in these areas plus the risk that "the project may still be unsuccessful in protecting natural areas and biodiversity due to local communities' poverty and lack of income alternatives". The Technical Advisory Councils, community outreach projects and links to other donors' projects were intended to mitigate this risk, although it was recognized that "while these steps are positive and desirable, the restructured project will be unable to fully address the pressures for social and economic development in the peripheries of the most populated reserves, and that it represents a starting point in this respect." However, in retrospect, it might be argued that more could have been done within the restructured project to meet this identified risk, which has turned out to threaten the achievements of the project in other areas.

Quality at Completion
Recipient and Bank performance during implementation appears to have been very good. Bank supervision was timely and staffed with appropriate expertise (the ICR says that it was "excessive" at times). Project implementation by the recipient was timely and of high quality. Government budgetary contributions exceeded appraisal expectations and permanent staff were assigned to each protected area. However, both sides focussed on the financing and institutional mechanisms that were the main concern at appraisal, with excellent results, but perhaps to the neglect of the biodiversity and social objectives. The PSR of 12/22/00 describes the results of the MTR. In addition to institutional and financial issues, social issues are covered but basically in terms of how to make the Technical Advisory Councils more effective. Biodiversity issues or achievements are not mentioned and there is no discussion as to whether the project design needed adjusting. As noted above, the ICR does provide limited information on biodiversity and social results, indicating but not analyzing significant problems.

Safeguards Compliance
Despite the fact that the Bank's safeguard policies were in place and well mainstreamed by 1997, there is no mention of them in the PD. Other documents show an EA Category of C, whereas B would have been more appropriate for a project with $1.6 million worth of small civil works. In addition, issues of project impacts on indigenous people and possible involuntary resettlement of populations within parks should have been discussed. However, indigenous peoples plans were developed and implemented. PSRs rate EA as Not Applicable, Natural Habitats and Indigenous People as Satisfactory and other policies as Not Applicable, without explanation. The ICR makes no reference to these omissions nor to any aspect of safeguards compliance. The Bank's performance in this area was clearly Unsatisfactory.

6. Ratings:ICROED ReviewReason for Disagreement/Comments
Outcome:
Highly SatisfactoryModerately SatisfactoryThe undoubted accomplishments in the institutional and financing areas need to be balanced with the "mixed" results in the primary biodiversity objective and the lack of information on the social objective.
Institutional Dev.:
HighHighSee 4. above.
Sustainability:
Highly LikelyHighly LikelySee 4. above.
Bank Performance:
SatisfactorySatisfactoryBut only marginally so, given the limited attention given to biodiversity and social issues and the lack of compliance on safeguards.
Borrower Perf.:
Highly SatisfactoryHighly SatisfactoryEven though the Recipient shares some of the blame for neglect of biodiversity and social objectives, the contrast of implementation performance with the 1992-7 period is so great that HS is warranted.
Quality of ICR:
Satisfactory

7. Lessons of Broad Applicablity:

1. Projects like this, which successfully take an integrated approach to protected area management plans, institutional strengthening, sustainable funding and the inclusion of local communities, are important vehicles for protecting globally valuable and threatened ecosystems.
2. Project components to involve local communities in conservation and to provide alternative income opportunities are essential if biodiversity objectives are to be achieved. However, such components need to be designed, supervised and monitored with care.
3. The endowment model can provide a powerful mechanism for ensuring a permanent flow of funds for operating protected areas, even in middle income countries but more so in the poorest countries. Such programs hold out the promise of sustainable management of protected areas, even if external donor funding should phase out.

8. Audit Recommended?  Yes

          Why?  1. Given the potential value of the endowment model for other countries, the evident success of the Mexico example should be fully analyzed.
2. The mixed results of the project in biodiversity and the unknown results of community development need further analysis, to see if there are positive longer-term trends.
3. GEF supports the recommendation.

9. Comments on Quality of ICR:

In conformity with the focus of the appraisal of the restructured project and Bank supervision, the ICR focuses on the considerable achievements towards the institutional and financing objectives, and is very adequate in these respects (apart from some repeated sections). Some useful lessons were drawn. However. it has only one descriptive paragraph on biodiversity outcomes and little information on the outcome of the community development activities. Tables of monitoring data on these outcomes, together with deeper analysis, would have been preferable. The absence of any discussion of safeguards compliance is another serious shortfall.

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions