Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Participatory Management Of Protected Areas


  
1. Project Data:   
ICR Review Date Posted:
04/11/2014   
Country:
Peru
PROJ ID:
P068250
Appraisal
Actual
Project Name:
Participatory Management Of Protected Areas
Project Costs(US $M)
 33.19  37.12
L/C Number:
Loan/Credit (US $M)
 14.80  14.80
Sector Board:
Environment
Cofinancing (US $M)
 0.00  0.00
Cofinanciers:
Board Approval Date
  03/13/2003
 
 
Closing Date
10/14/2009 12/31/2010
Sector(s):
Other social services (45%), Forestry (40%), Central government administration (10%), Sub-national government administration (5%)
Theme(s):
Biodiversity (25% - P) Participation and civic engagement (25% - P) Environmental policies and institutions (24% - P) Rural non-farm income generation (13% - S) Gender (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Richard C. Worden
Ridley Nelson Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
The Project Development Objectives (PDOs), as stated in the GEF Trust Fund Grant Agreement (Schedule 2, p. 25), were:


    to ensure biodiversity conservation by (i) by increasing the involvement of civil society institutions and the private sector in the planning and sustainable management of at least five of the national protected areas (Areas NaturalesProtegidas, ANPs) of the National System of Protected Areas in Peru (SINANPE), and by creating at least one additional protected area during Project implementation; and , (ii) to obtain sustainability for the financing of recurrent costs in SINANPE.”
This is identical to the definition of the PDO in the Project Appraisal Document (PAD, p.2), The Global Environment Objective (GEO), as defined in the PAD (ibid), was: “to contribute to the conservation of biodiversity of global importance [in Peru].

For the purposes of this review, the PDO as stated in the GEF Grant Agreement will be used.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The Project had three components:

1: Participatory Protected Area Management (Appraised: US$18.23 M; Actual: US $19.51 M.) The objective of this component was to allow civil society organizations and local community and private sector organizations to participate actively in decision-making processes and management of the ANPs and share in the benefits resulting from the ANP sustainable use.

2: Institutional Development (Appraised: US$5.74 M; Actual: US$4.93 M.) The objective of this component was to consolidate the administration of ANPs by: (i) increasing the capacity of the National Institute for Natural Resources (INRENA), the Peruvian National Trust Fund for Protected Areas (PROFONANPE), and the Management Committees (CG) of individual ANPs to administer and manage ANPs in participation with civil society and the private sector; (ii) strengthening local civil society organizations’ capacity to support Project objectives; (iii) inducing an environmental behavioral change through increased conservation awareness; and (iv) supporting an enhanced decision-making process by national administrators, stakeholders and civil society organizations through better and updated information on ANP management and biodiversity.

3: ANP Financing, Administration, Monitoring and Evaluation and Information Dissemination (Appraised: US$8.29 M; Actual: US $12.43 M.) The objectives of this component were to increase PROFONANPE’s Endowment Fund, design and implement a financing strategy for SINANPE, manage and monitor the Project, and disseminate information on the Project’s progress and achievements.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: Total project cost was US $37.12 million, up nearly four million dollars over the US $33.19 million that was estimated at appraisal. The cost of preparing participatory plans for managing ANPs was over-budget by US$9 million, or more than twice as much as expected at the outset of the project (US$17.27 million actual cost vs. US$8.15 million at appraisal), and increased M&E added another US$3.30 million. Both of these increases were offset in large part by a large drop in expenditures to implement sustainable economic activities in ANPs and buffer zones of US$5.83 million.


Financing: The project was financed by a GEF grant of US $14.80 million (as proposed at appraisal). There was parallel-financing assistance received from the governments of Finland (US $2.50 million actual and proposed) and the Netherlands (US $4.44 million proposed and US $2.05 million actual). The German government’s KfW provided US $5.58 million in parallel financing out of US $6.58 million that was proposed at appraisal.

Borrower Contribution:US $3.16 million was provided by the Peruvian government to the project, US $200,000 over its agreement to contribute US $2.96 million. In-country support was received for US $530,000 from local farmers associations, and for US $8.25 million from Peruvian NGOs (up substantially from an estimated US $1.00 million at appraisal).

Dates: The project was restructured twice (first on October 15, 2007 and then again on April 29, 2011). It was originally slated to close on October 14, 2009, but was extended just over a year and closed on December 31, 2010. The 13-month extension was caused by strong opposition to the regulatory pre-conditions governing use of donor funds that delayed their use by 18 months.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial.


Project objectives were relevant at entry and at closing. The objective of ensuring biodiversity conservation by: (i) by increasing the involvement of civil society institutions and the private sector in the planning and sustainable management of ANPs, and (ii) to obtain sustainability for the financing of recurrent costs in SINANPE were relevant to the Bank’s current Country Partnership Strategy (for FY 2007 – 2011). Objectives are also relevant to Peru’s own national development plans that include new environmental laws and institutions, such as the creation of a new National System of Environmental Management and the General Law of the Environment enacted in 2004 and 2005 respectively, and the creation of a new Ministry of the Environment (MINAM) and National Protected Areas Service (SERNANP) in 2008. The project was also relevant to the GEF’s Operational Programs for coastal, marine, and freshwater ecosystems (OP2), forest ecosystems (OP3), and montaine ecosystems (OP4) that supported GEF’s strategy for biodiversity conservation and the principles of the Biological Diversity Convention which Peru signed.

b. Relevance of Design:
Modest.

The PAD identified four “sector threats and constraints” that negatively affected the country’s ability to protect and conserve biodiversity of global importance at the start of the project: (i) limited public sector capacity to design and implement policies in a collaborative manner with civil society organizations and the private sector; (ii) limited financial resources and the lack of a self-financing strategy and mechanisms to reduce over-dependence on international donors and NGOs; (iii) lack of environmental awareness in general and to conserve biodiversity more specifically among the general public; and (iv) inadequate management information systems (MIS) to manage protected areas, promote greater public awareness, and inform decision-making processes. The project’s design responded to each of these four constraints by including specific activities and outputs to address them.

The three components and sub-components of the project design were linked to the global and project development objectives in most cases, but the involvement of CSOs and the private sector via Administrative Contracts in the planning and sustainable management of ANPs to improve their “management effectiveness” does not necessarily "ensure biodiversity conservation" although it may contribute to that goal. Nor does the sustainable financial support provided to the national system of parks and reserves (SINANPE) provide a clear and direct causal chain to ensuring biodiversity conservation although it also has potential to contribute. Finally, the project relied too heavily on adopting a monitoring and evaluation (M&E) system developed under another predecessor project (the Indigenous Management of Protected Areas in the Peruvian Amazon (PIMA) Project) without adequate, prior verification of its applicability and utility to this Project.


4. Achievement of Objectives (Efficacy) :

The two Project Development Objectives were: “(i) to ensure biodiversity conservation by increasing the involvement of civil society institutions and the private sector in the planning and sustainable management of at least five of the ANPs of SINANPE, and by creating at least one additional ANP during Project implementation; and (ii) obtain sustainability for the financing of recurrent costs in SINANPE.”

a) PDO #1: Ensuring biodiversity conservation by increasing the Involvement of Civil Society Institutions and the Private Sector in the Planning and Sustainable Management of ANPs, and by creating at least one additional ANP. Rating: Substantial.

The first part of the PDO was to ensure biodiversity conservation. This was to be achieved through two intermediate outcomes: involving CSOs and the private sector in the planning and management of five national parks (ANPs) in the system of national parks and reserves (SINANPE) in Peru, and by adding one more park to the system.

The monitoring and evaluation (M&E) system of the Indigenous Management of Protected Areas in the Amazon (PIMA) Project was intended to be the key instrument to monitor and evaluate this project’s impact on biodiversity conservation and protection. PIMA’s M&E system, however, proved inadequate to that purpose as its indicators were “not suited to the technical and scientific requirements of the Project” (ICR, p.12). Instead, the project attempted to use biological monitoring data collected by implementing regular monitoring and evaluation protocols on the status of 25 biological resources in the targeted ANPs, as stipulated in the Contracts. However, even after a project of over seven years, there were very few direct measures of improved biodiversity of global significance presented in Annex 2.B of the ICR (pp.41-44). There was an increase in numbers of wild vicunas by 22.7% against a target (set over the shorter than actual planned project life) of 17%. and the population of semi-captive vicuñas in Salinas y Agua Blanca National Reserve rose by 34%, somewhat below the target of 40%. Other than these few data points, no other direct evidence about the practical effect or impact of the project was presented in the ICR. Predominantly indirect measures of outputs (Table 4, pp.16-18) were shown to assess the achievement of the PDO, such as the number of programs implemented in five ANPs to reduce biodiversity threats, or the number of professionals conducting biological monitoring, sustainable agriculture, water resources management, and surveillance planning, who were employed by contractors.

However, the establishment of the processes and institutional linkages was important and the increased involvement of civil society and the innovative private sector element of this objective appears to have been achieved. In terms of outputs, the Project was eventually successful in meeting all of its indicator targets for the first PDO:

· Two new and four updated ANP management plans were developed,
· Five community groups were operating in the project-supported APNs, and
· Three Administration Contracts were signed with concessionaires in four ANPs (all), overcoming initial resistance, delays, and distrust among the public and affected communities. The attempt to create an Administrative Contract in the fifth ANP (Huascarán National Park) was dropped “due to violent resistance by illegal tourism operators and others” (ICR, pp. 10-11).
· According to the ICR (p. 21), approximately 10 Administration Contracts are currently under bid or have been awarded in other ANPs due to the successful experience of the Project, and SERNANPE is “rolling out the approach across the entirety of SINANPE” [the system of protected areas in Peru] due to the “demonstrated effectiveness of the participatory management approach supported by the Project.”
· Women constituted more than one-third of the beneficiaries of the 62 small-scale sustainable economic activity sub-projects implemented by the Project. The target set in the PAD was 10 percent.
· Approximately 30 percent of the 57 CSOs and non-profits participating in those pilot projects were indigenous groups representing 25 percent of its beneficiaries (no PAD target).
· Women and indigenous groups also made up 30 and 25 percent, respectively, of the members of the local management groups (CGs) that were established by the Project in the five ANPs.

Finally, the new ANP that was added to SINANPE’s system (Morona Pastaza) was not directly attributable to the project since it was funded by the Kreditanstalt für Wiederaufbau (German Financial Cooperation Agency, or KfW) with parallel financing, and not by this project.

The Efficacy of this first PDO to ensure biodiversity conservation by increasing the involvement of civil society organizations and the private sector is assessed on balance as substantial based on expectation of future impact.

b) PDO #2: Obtain the Sustainability for the Financing of SINANPE’s Recurrent Costs. Rating: Substantial.
The wording of the second PDO was not clear in terms of indicating what measure or standard should be used to objectively assess the degree to which it was achieved. On the positive side, the Project made a substantial contribution to enhancing SINANPE’s ability to sustainably finance its annual recurring operating costs. This was due to multiple factors, including:

· US$6 million in grants from The GEF and other donors to PROFONANPE, which more than doubled its endowment from US$5.4 million to US$11.4 million. This allowed it to increase its annual contributions to SINANPE’s recurrent costs with realized yields of US$1.4 million accrued during project implementation.

· Additional resources leveraged by the Project through the use of Administration Contracts included US$10 million from three private sector entities in the four ANPs that the Project supported (ICR, p. 17), and an additional US$15 million that municipal, local, and regional governments and international donors provided to support biodiversity conservation and sustainable alternative economic activities (PAES interventions) in those four project-supported ANPs and their buffer zones. This total of US$25 million in “other” financing support was nearly an order of magnitude greater than the output target set at project approval to “capture US$3 million in additional funds over the life of the project” (PAD, Annex 1, pp. 33-35).

· Modifications to the regulatory framework governing the contracts were made, which allowed all financial resources generated from tourism permits, natural resource use license fees, and other ANP goods and services to be retained by protected areas system rather than to flow into the national Treasury.

· Increased revenues gained from the improved management of the ANPs (for example, through better collection of tourism permits, natural resource use license fees, and other ANP goods and services provided). By Project closing, US$1.2 million in annual revenues from such revenue sources was being collected, roughly equal to 50% of the total permit and licensing revenues collected by the other 50 units of SINANPE’s ANPs (ICR, Table 4, p. 19). The project’s successful approach is being scaled up with the intention to extend it to 15 other ANPs (ICR, p. 22).

· Continued good performance of PROFONANPE’s asset management of its portfolio holdings (with their above-market average returns obtained in all but one year over the last decade), and the preparation and approval of SINANPE’s new 10-year financial strategic plan have attracted new sources of financial support and prompted the World Bank and other donors to approve the PRONANP Project, a follow-on operation to this project that started in November and adopted this project’s approach to manage other ANPs.

On the negative side, these new revenue sources and increased contributions to SINANPE’s recurrent operating costs have not significantly reduced SINANPE’s annual shortfall of approximately US$3.25 million. However, in the absence of such support (the counter-factual), that shortfall would likely have been much greater, although it is not possible to estimate this precisely.

While the degree of attribution to the Project was not clearly indicated in the ICR, it was evident that improvements made to the ANPs and buffer areas were largely motivated by the presence of the concessions granted by these Administration Contracts as well as by the tangible benefits of the Project’s sustainable economic programs and activities. In its Closing Report, PROFONANPE stated that the Administrative Contracts had “contributed to improved research and biodiversity monitoring within the targeted protected areas, increased both short-term financing for some parks, had partially closed the system’s operating cost gap, and set a course toward greater long-term financial sustainability of the system” (ICR, p. 53). Thus, it is assessed that the second objective was substantially achieved.

c) Progress toward the Global Environmental Objective of contributing to the conservation of biodiversity of global importance in Peru. As noted above, only limited indirect evidence was presented in the ICR (Annex 2B) regarding improved biodiversity conservation (BDC) results. There were a number of indicators presented throughout the ICR that would likely lead to greater BDC, such as increased financial support, greater public awareness about and participation in BDC efforts, and improved management of protected areas representing different types of biodiversity but there were only a few direct measures of improved biodiversity of global significance presented in the ICR or provided subsequently. There was little evidence in the form of field surveys or threat reduction plans being implemented by the project, thus the achievement of the project’s Global Environmental Objective depends largely on expectations that the processes and institutional linkages established, some of which are innovative, will lead to that outcome.


5. Efficiency:

There was neither an economic or financial analysis done at appraisal given the non-market nature of the Project. As is normal for GEF-funded projects, an incremental cost analysis was done showing a difference of US $32.8 million between the “no-Project” baseline scenario of US $13.3 million and the total financing costs of the GEF alternative (US $46.1M), representing the incremental costs associated with the proposed additional Project activities and investments required to secure biodiversity conservation benefits of global importance.

Since traditional efficiency measures in this case give little indication of how efficiently Project funds were used to achieve the Project’s objectives, the assessment is based on other implementation findings as to whether the Project used its resources to achieve a reasonable value for the expenditure. These include the following:
  • There were a number of changes in the Project’s execution that slowed down implementation or negatively affected expected outputs and outcomes. For example, there was a formal amendment made to the GEF Grant Agreement in 2003, soon after the effective date, to reduce the number of sustainable alternative economic activities (PAES) by half.
  • Less than a quarter of the funds programmed to implement PAES activities as well as to prepare master plans and management plans for individual protected areas were actually spent.
  • Conversely, the amount of resources needed to complete the participatory implementation plans and Administrative Contracts was more than double of that planned while the number of Administrative Contracts was reduced from five to four due to violent opposition in one protected area.
  • Not only was opposition to these legal instruments much stronger than estimated initially, but the regulatory pre-conditions governing use of donor funds was delayed by 18 months. Thus, they were very time- and labor-intensive to finalize, requiring Project funds to be reallocated and resulting in delays and a 13-month extension of the Project.
  • Finally, and probably the most important source of inefficiency in executing the Project, the attempt to adopt the monitoring and evaluation (M&E) system from another Bank project (the PIMA Project) led to delays and inefficiencies. These “proved to be inadequate for accurately reporting on the achievements of participatory APN [protected areas] and biodiversity conservation management” (ICR, p. 9). As a result, another M&E system to monitor the status of key species in the five protected areas targeted by the Project had to be developed for use under the Administrative Contracts: a “complementary methodology was developed to be applied directly by the Project technical team to obtain information from the Project’s beneficiaries and other stakeholders to assess their participation” (ibid). The costs to implement Sub-component 3.3 (Project Implementation and the M&E System) were US $3.5 million or 235 percent above those projected at appraisal.

Given the limited degree of impact that the additional funds made to the PROFONANPE trust fund relative to its needs to cover basic operating costs (decreasing the annual “funding gap” of US $3.25 million by less than US $200,000), and the unexpected challenges posed by the PAES activities, Administrative Contracts, and the M&E system, overall efficiency is assessed as Modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The Relevance of Objectives was rated as substantial due to the strong linkage to the Bank’s Country Partnership Strategy (FY 2007 – 2011) in effect at the time of project closure. The Relevance of Design was assessed as modest due to the lack of a clear, logical causal chain between the design of the project’s Results Framework and achieving the PDO. The Efficacy of the first PDO, to ensure biodiversity conservation by increasing the involvement of civil society organizations and the private sector, is assessed on balance as substantial based on expectation of future impact. The second PDO was assessed as substantial because project efforts and resources resulted in significant leveraging of civil society and private sector resources through the use of concession contracts to address the chronic shortfall in financing the annual operating budgets of the protected areas system. Efficiency of project implementation was rated as modest due to a number of implementation inefficiencies. Overall the project had moderate shortcomings.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The likelihood of sustained involvement of civil society institutions and the private sector in the planning and sustainable management of ANPs is fairly high. The use of Administrative Contracts that was promoted by the Project in four protected areas is now being used in four more ANPs with longer terms to encourage non-profits to make greater investments in the concession of the protected area. Six new bids for 20-year concessionaire contracts are under negotiation with private sector non-profit organizations, and all 10 of these Administrative Contracts are expected to be in effect by the end of 2011 (ICR, p. 47). None of these contracts has been made with any of the 11 non-profits originally identified in the appraisal process, demonstrating that this mechanism has broadened the private sector’s biodiversity conservation constituency base.


The sustainable alternative economic activity approach used by the project to discourage unsustainable extractive activities in ANPs and their buffer zones is now being extended to an additional 15 protected areas (out of 54 ANPs in all) by SERNANP. According to the ICR, there are two follow-on German-financed KfW projects scaling up the approach initiated by the Project, and SERNANP has plans to extend it to the entire protected areas system (p. 15), although this must be done carefully to avoid the public’s concerns about privatizing public goods and services.

Institutionally, Peru has gained experience in structuring and supervising Administrative Contracts with private sector non-profits as a result of all of these projects, and has upgraded its regulatory framework and relevant organizations (having created both a new environmental ministry (MARN) and a protected areas service (SERNANP) in 2008. SERNANP has developed mechanisms to improve coordination and information sharing among the protected areas’ management committees. One of these is the National Council for the Coordination of Protected Areas Management Committees (CNCCG), which has become one of Peru’s broadest institutional platforms with over 1,000 stakeholders, and serves as one of the principal drivers of participatory ANP management to conserve and protect biodiversity in Peru. There is little chance this empowering process will be reversed.

In terms of ensuring the financial sustainability of SINANPE’s recurrent costs, this objective is still at risk to the possibility that foreign donor support could dry up. SINANPE’s finances are still heavily dependent on this less stable source of financial support, particularly among bi-lateral donors such as German, Finnish, Canadian and American government aid agencies. The majority of donor funds supporting SINANPE are a combination of sinking and endowment funds (unlike the GEF fund which is strictly an endowment-type fund), accounting for well over half of PROFONANPE’s current capital assets of US $29 million.


The current capitalization of PROFONANPE is just over half of the amount identified in the Project’s Financial Strategic Plan of US $55.35 million needed to close the annual operating cost gap of US $3.5 million between SINANPE’s basic needs and current budget. The Financial Plan includes an additional US $6.5 million from the GEF to supplement the less than half million dollars they contribute to the system’s current operating budget of US $9.5 million. The Project contends that private sector non-profits, NGOs, and other sub-national government entities have supplemented this shortfall, which is true, but it still falls far short of what is needed to bridge that gap, and it is narrowly focused on just a handful of the 54 ANPs in the national protected areas system.

At appraisal, 43 ANPs were “Category I” protected areas, meaning they receive little to no funding (defined as less than US $100,000/year) and were “not capable of achieving basic biodiversity conservation objectives” (PAD, p. 97). Even the US $9.5 million funding “goal” proposed in the Gap Analysis conducted at the start of the Project was “characterized as one where basic visitor and community outreach services could be provided in 40 of the 54 ANPs and biodiversity conservation would be assured in all ANPs” (ibid). As stated in the ICR following the closing of the Project, Peru’s dependence on donors and NGOs to finance recurring operational costs “has not changed fundamentally” (p. 23).

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
The design of the participatory arrangements which were the central element in the entire protected areas strategy and the foundation on which improved outcomes in biodiversity conservation have been built” (p. 25). This statement was reiterated and confirmed by both the Borrower’s comments in its “closing report” and those of the main co-financing German aid agency (KfW) which characterized participatory management as a “fundamental aspect of success in the management of ANPs in Peru” (Annex 7; p. 53 and Annex 8; p. 54, respectively). The Project made substantial progress in its emphasis on strengthening Peruvian government institutions and civil society organizations, such as the management committees in the four targeted ANPs, as well as for its work to improve the effective management of these protected areas through the program of alternative sustainable economic activities (PAES) to reduce threats to protected areas and improve local livelihoods even though the evidence is insufficient that these contributed directly toward the achievement of the project’s development objectives as specified. Preparations made to ensure compliance with the Bank’s environmental and social safeguard policies as well as with its fiduciary requirements were carefully and fully carried out by the Bank’s project staff.

There were, however, several serious flaws and shortcomings in the Bank’s performance at entry:

  • First, given the importance of the Administrative Contracts in achieving the project’s development objectives to increase the participation of local non-governmental stakeholders and as a means to increase capital infusions in the management of selected protected areas, the intense resistance that they generated among various important stakeholders should not have caught the Project by surprise. Mitigation strategies should have been prepared in anticipation. Related to this was the fact that the Project was caught unaware by the lack of regulatory authority allowing external donor funding to be used to establish Administrative Contracts, although much of the blame for this lapse belongs with the borrower’s implementing agency for not alerting the Bank to this impediment beforehand.
  • Second, it appears that the preparations for implementing the program of alternative sustainable economic activities were not adequately considered. This necessitated a change in the scope and types of PAES activities undertaken. The fact that the Bank, implementing agencies, and grant recipient all agreed that there were too many activities planned given weak local capacities and limited Project resources was simply evidence of this rather than a reformulation of PAES activities in the face of changing conditions that could not have been known previously.
  • Third, a biodiversity and protected areas monitoring and evaluation (M&E) system was developed under a predecessor project (Indigenous Management of Protected Areas in the Peruvian Amazon (PIMA) Project). This project planned to adopt the M&E system once it had been field-tested under the PIMA Project as the primary means of keeping the public better informed about its activities/achievements, and building more trust in local institutions through greater transparency and clarity of decision-making processes. However, this did not occur as the PIMA M&E system was found to be “inadequate” to the needs of the project and other mechanisms had to be employed to collect and report project information. This situation might have been avoided with a more careful assessment of the PIMA M&E system before positing so much confidence in such an important Project output.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
The Bank’s supervision is considered satisfactory. First, the Bank’s Project staff worked diligently with the implementing agency (INRENA) to resolve the legal barriers prohibiting the use of Project funds to establish and fund the Administrative Contracts with private sector non-profit organizations. Once this obstacle was overcome, Project staff then incorporated adequate institutional coordination arrangements and legal covenants into the contracts to include biodiversity M&E activities to track the status of key species. When the Dutch Embassy informed PROFONANPE in 2003 that the US $4.4 million intended for the Project could only be used to support PAES activities, the Bank coordinated with PROFONANPE and the German aid agency (KfW) to cover the activities that the Dutch were going to fund through another parallel project. When PROFONANPE needed more time to overcome legal obstacles and public resistance to implementing the novel approach of co-management arrangements with private sector non-profits, the Bank’s staff sought and received formal approval from Bank management to extend the project for another year in order to have sufficient time to allow the Administrative Contracts to mature. When the Project encountered local protected areas administrative bureaus that were technically weak and management committees that were marginalized and distrustful of INRENA’s intentions, Project staff spent considerable time and effort into improving their relations and capabilities to the point where SERNANPE acknowledged in its Closing Report that the committees had been strengthened as a key mechanism for participatory management with M&E activities being jointly conducted by SERNANPE staff and committee members.

Second, due to its persistence in pursuing the implementation of the Administrative Contracts with non-profits, the Project was able to demonstrate the effectiveness of well-planned and supervised arrangements for co-managing protected areas. Finally, the Project revamped the communication strategy, which had not differentiated between different stakeholder groups, to promote greater transparency and trust between them that was critical to , engaging local stakeholders in the planning and management of those protected areas according to both SERNANP and the German aid agency KfW in their comments summarized within the ICR (Annex 8, p. 54).

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government met its financial obligations as the Borrower, contributing US $200,000 more than the appraisal estimate of US $2.96 million to the Project. In addition, it took steps to upgrade the institutional base of relevant government organizations during the Project, such as creating a new Ministry of the Environment and a National Service of Protected Areas (SERNANPE).


The Government’s failure to “have anticipated or at least identified all the administrative steps to be taken in order to execute the Administrative Contracts, and to enable local [private sector non-profit] organizations to participate in the Program of Sustainable Economic Activities”(ICR page 27) caused a delay of 18 months in the use of project funds for this purpose, but eventually non-profits were registered and allowed to participate in the PAES pilot program.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
SERNANPE successfully developed the necessary legal conditions and institutional coordination to establish and manage the pilot protected areas in accordance with the Bank’s requirements for accountability and transparency, and selected and monitored those non-profits thereby meeting the targets established. This was accomplished despite initial resistance to the novel idea of co-managing these publicly held protected areas with non-profits from the private sector and after overcoming difficulties encountered in the process of implementing them in the four pilot ANPs, SERNANPE also completely turned around the situation that existed between local stakeholders and its own local staffs by opening up the management decision-making process to greater participation and empowerment by local affected parties. They began financing the participation of local management committees in the national strategic and operational planning process for protected areas, a process that previously had included only SINANPE park administrators, which was a major step forward. By mandating annual meetings with Administration Contractors and the presidents of these management committees regarding the Contractors’ annual operating plans and their respective responsibilities, it introduced much greater transparency and reduced the potential for social conflict. As the recipient of the GEF grant, PROFONANPE executed its responsibilities for procurement and financial management competently and provided timely reporting in line with the Bank’s requirements.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The M&E system developed by the PIMA Project was not used since it did not meet the Project’s scientific biodiversity conservation needs and requirements. Instead an alternative M&E system was developed by the Administrative Contractors in the four ANPs as a condition of their concessionaire legal obligations. These were developed with input from more than a dozen universities and research facilities, management committees representing local interests, and contractors hired for this purpose. They contained data for each of the three broad objectives of the protected areas’ master plans: biodiversity conservation status on more than 20 key species, sustainable natural resource use, and measures of financial sustainability. Information on Project activities directed toward beneficiaries and the degree of stakeholder participation was gathered and used to generate data on the number and types of beneficiaries (indigenous peoples, women, and local membership in the management committees) and their experiences. These were summarized in Annexes 2B (pp. 41-44) and 8 (p.53) of the ICR.


The three key performance indicators under the first PDO to increase the participation of stakeholders, women, and indigenous peoples also overlapped with one another (resulting in double- and triple-counting of some indicators) and lacked baselines and targets to measure the degree of achievement of those key performance indicators. Therefore, there were some redundancies and unnecessary key indicators in the causal chain of the Results Framework from inputs and activities to outputs and outcomes that obscured their relevance to achieving the project’s objectives.

b. M&E Implementation:
The M&E system developed by the PIMA Project was not used since it did not meet the Project’s scientific biodiversity conservation needs and requirements. Instead an alternative M&E system was developed by the Administrative Contractors in the four ANPs as a condition of their concessionaire legal obligations. These were developed with input from more than a dozen universities and research facilities, management committees representing local interests, and contractors hired for this purpose. They contained data for each of the three broad objectives of the protected areas’ master plans: biodiversity conservation status on more than 20 key species, sustainable natural resource use, and measures of financial sustainability. Information on Project activities directed toward beneficiaries and the degree of stakeholder participation was gathered and used to generate data on the number and types of beneficiaries (indigenous peoples, women, and local membership in the management committees) and their experiences. These were summarized in Annexes 2B (pp. 41-44) and 8 (p.53) of the ICR.

a. M&E Utilization:
The data collected from the M&E systems and generated by the Administrative Contractors in quarterly and annual reports were reviewed and approved by the local SERNANPE park administrators and stakeholder management committees to assure the quality of the information. PROFONANPE participated in reviewing the financial and technical aspects of these M&E reports. Although it was not explicitly laid out in the ICR how the information generated by the M&E systems for the Project affected decision-making processes and the execution of Project activities, it was apparent from the ICR that the information was being used by government entities as well as by the Administrative Contractors and local management committees to fine-tune and adjust planning and management of the ANPs as an on-going process that continues to this day. This is evidenced by the fact that the National Council for the Coordination of Protected Areas Management Committees (CNCCG), whose establishment and operation the Project supported, has taken on the role of promoting at the national level the development of participatory management and use of Administration Contracts in other protected areas of SINANPE.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
Four Safeguard Operational Policies were triggered by the Project, which were: OP 4.01 (Environmental Assessment), OP 4.10 (Indigenous Peoples), OP 4.12 (Involuntary Resettlement), and OP 4.36 (Forestry). The Project was classified as a Category “B” under OP4.01 Environmental Assessment by the Bank in its assessment of potential environmental and social impacts. Due to a number of small-scale, but undefined, infrastructure projects associated with the alternative sustainable economic activities program, an Environmental Framework was put in place and was supported by an Environmental Management Program to determine when and which mitigation measures would be required, as well as a Process Framework and Indigenous Peoples’ Development Plan developed to support the Project’s Social Strategy to mitigate negative social impacts. All of the impacts generated by the Project were successfully mitigated. According to the ICR (pages 13-14) project implementation complied satisfactorily with environmental and social safeguards including Forestry and Natural Habitats where there was some uncertainty about whether Forestry was triggered.

b. Fiduciary Compliance:
The Project complied with all of the fiduciary requirements of the Bank in terms of procurement and financial management. The Project Implementation Unit’s procurement and contract administration processes were accurate, reliable, timely, and transparent. PROFONANPE successfully implemented an Action Plan that responded to recommendations from the Bank’s procurement reviews. As a result its selection and supervision procedures for the Administration Contractors’ execution of their procurement responsibilities improved during Project implementation. This was facilitated by the creation of a dedicated financial management unit for this sole purpose, which established and maintained adequate financial management arrangements to support Project implementation. The FM unit also developed the expertise to control transfer of funds to the ANPs effectively, an inherent FM risk that had been rated “modest” at Project appraisal. All audit reports received, including the final audit, were submitted on time and were unqualified.

c. Unintended Impacts (positive or negative):
None encountered.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
Efficiency was rated modest due to a number of implementation inefficiencies.  
Risk to Development Outcome:
Negligible to Low
Moderate
PROFONANPE’s still limited financial self-sufficiency continues to pose a moderate risk to the protected areas system’s effective conservation. 
Bank Performance:
Satisfactory
Moderately Satisfactory
There were weaknesses in Quality at Entry due to insufficient preparedness for resistance by some stakeholders, weakness in preparation for sustainable economic activities and problems with M&E planning and design.  
Borrower Performance:
Satisfactory
Moderately Satisfactory
Poor implementation of administrative regulations/procedures resulted in an 18 month delay in the implementation of Administrative Contracts with NGOs and the private sector in the pilot ANPs. 
Quality of ICR:
 
Unsatisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following broad lesson is drawn from the ICR:

A long-term, results-oriented management approach through Administrative Contracts with the private sector may promote efficiency better than a more traditional, shorter-term, project-specific management contract. The unique aspect of this Project was the approach it took to use Administrative Contracts as an innovative mechanism to co-manage protected areas with private sector non-profits and with input from key local stakeholders. Not only did this function to broaden local participation and empowerment with government natural resources management authorities, but it also proved to be an effective tool for expanding the government’s limited absorptive capacity by complementing it with untapped pools of private sector expertise and resources. Given the longer-term concession terms of 20 years, these legal instruments shifted the short-term perspective of the more typical project-specific management arrangements to a longer-term results-oriented management approach that encouraged greater capital investments by these private sector actors.

14. Assessment Recommended?

Yes
Why?
To verify the ratings -- to assess the extent to which the processes established that promise achievements on biodiversity are actually leading to the biodiversity outcomes anticipated. Also, among the four broad topics covered under “Lessons Learned” in the ICR, the one aspect of this Project that IEG found unique was the approach it took to the use of Administrative Contracts as a mechanism to co-manage protected areas with private sector for-profit companies with input from key local stakeholders. This is worth deeper exploration for Bank learning.

15. Comments on Quality of ICR:

On a positive note, the ICR did provide useful information on the views of various stakeholders, including the borrower, implementing agency, co-financiers, and project beneficiaries. The ICR covered all required subjects and did so comprehensively, if not somewhat redundantly, and the sections on “Risks to Development” and “Lessons Learned” also provided information that was insightful and candid.

On the other hand, there was little direct evidence provided in the ICR substantiating the assertion that the project was “largely successful” in achieving its broad objectives, especially at the PDO and GEO level. Instead, its reporting focused on outputs and key performance indicators, several of which had little bearing on the extent to which either the GEO or PDO was achieved. The task of assessing the efficiency of the project’s implementation was further complicated by the confusing way in which much of the financial data was presented in the ICR. For example, it was difficult to ascertain which financial assistance was co-financing project activities and which support was parallel financing—separate but complementary to project activities. Little useful information was provided regarding project implementation efficiency, making the process of assessing the project more difficult. The ICR did not adequately explain what impact the project’s contributions to PROFONANPE had on SINANPE’s financial sustainability. In addition, by focusing so much on the design problems associated with the proposed M&E system developed by the PIMA Project, there was too little information provided regarding the implementation and utilization of the M&E system that was eventually used for this project. Finally, insufficient direct indicators or measures of actual biodiversity conservation results were provided.


a. Quality of ICR Rating: Unsatisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions