Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Brazil - Ecosystem Restoration Of Riparian Forests In Sao Paulo

1. Project Data:   
ICR Review Date Posted:
Project Name:
Brazil - Ecosystem Restoration Of Riparian Forests In Sao Paulo
Project Costs(US $M)
 19.52  21.77
L/C Number:
Loan/Credit (US $M)
 7.75  7.75
Sector Board:
Cofinancing (US $M)
Board Approval Date
Closing Date
01/31/2010 04/27/2011
General agriculture fishing and forestry sector (40%), Agricultural extension and research (20%), Sub-national government administration (20%), Forestry (10%), Other social services (10%)
Environmental policies and institutions (25% - P) Biodiversity (25% - P) Participation and civic engagement (24% - P) Water resource management (13% - S) Land administration and management (13% - S)
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Ridley Nelson
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
This was a stand-alone GEF project. The Global Environment Objectives (GEOs), as stated in Project Appraisal Document (PAD, page 7) were: "to arrest and reverse land degradation processes in riparian ecosystems and adjacent agro-ecosystems by increasing on-the-ground investments and strengthening the policy, regulatory, economic, and institutional incentive framework to encourage sustainable land management, hence increasing carbon sequestration and restoring ecosystem stability, functions and services".

b. Were the project objectives/key associated outcome targets revised during implementation?

c. Components:
Component 1. Policy Development. (Appraisal US$1.67 million; Actual US$0.92 million). This was to establish realistic legal, technical, financial and economic frameworks for the future implementation of a statewide riparian forests restoration program.

Component 2. Support to Sustainable Riparian Forest Restoration. (Appraisal US$1.76 million; Actual US$1.05 million). This was for the development and field testing of techniques for riparian forest rehabilitation and restoration and improved market supply of native seeds/seedlings to achieve long-term restoration goals.

Component 3. On-the-Ground Investments in Sustainable Land Management (SLM) Practices. (Appraisal US$10.38 million; Actual US$12.73 million). This was to finance the promotion and dissemination of tested SLM practices, including zero till agriculture and terracing, and to pilot restoration activities through investments in selected micro watersheds.

Component 4. Environmental Education and Training. (Appraisal US$2.52 million; Actual US$1.06 million). This was to establish the basis for participation of local populations in planning and implementing local/regional development and conservation activities focusing on better quality of life from the use of SLM.

Component 5. Project Management, Monitoring and Evaluation, and Information Dissemination. (Appraisal US$3.19 million; Actual US$6.01 million). This was to finance the coordination, management and monitoring of project activities at the state, regional and national levels.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
There was an approximate doubling of the project management costs, and some reduction in the costs of the policy development, sustainable restoration, and environmental education components.

The Borrower provided US$14.02 million, approximately double the financing from GEF and about 20% higher than planned at appraisal. The project was initially intended to be an IBRD/GEF blended operation. However, the State had reached its borrowing ceiling. It was therefore proposed that it would be a pure GEF project but that the Borrower contribution would include US$7.34 million of parallel financing from the Land Management Project III covering the state of Sao Paolo.

There was an amendment to the Grant Agreement, a Level II restructuring, effective December 16, 2009 to: (i) add three new definitions related to the consultant and procurement guidelines and the project procurement plan; (ii) reduce the number of seed production centers from two to one; and, (iii) reduce the area of sustainable riparian forest rehabilitation from 1,500 ha to 500 ha (one of the key indicators).

The closing date was extended by 15 months from January 31, 2010 to April 27, 2011 to enable completion of key activities.

3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Rated Substantial. The GEOs were consistent with the 2008 - 2012 Country Partnership Strategy (CPS) at project closure, in that they contributed to a goal of improved water resources management, sustainable land management, and biodiversity. The CPS specifically discusses sustainable forest management and the project supported the Environment and Natural Resource Management pillar of the earlier Country Assistance Strategy.

The GEOs were also consistent with the GEF Operational Strategy and the Operational Program for Sustainable Land Management (OP 15). They were relevant to GEF Operational Policy Statement (OP 3) on Forest Ecosystems.

b. Relevance of Design:
Rated Negligible.
Design was inconsistent with the ambitious objectives statement. Arresting and reversing degradation is a slow process. Many of the activities financed, and the corresponding output and outcome targets, were inappropriate to achieving the Global Environmental Objectives as stated. There was insufficient time for results to appear from the planned interventions, particularly treatments such as the planting of indigenous species that take many years to mature.

The ICR argues that the fundamental intent was to provide the State of São Paulo with the capacity and tools to tackle future restoration by piloting approaches and establishing an appropriate legal and technical foundation. This is not what the objectives state.

Design presented other shortcomings that would have made it difficult to achieve objectives. First, as noted by the ICR, expecting that the pilot testing would present lessons in time for scaling up was too optimistic. Second, the overall integration needs presented difficult challenges and, again, the phasing in that respect was optimistic. Third, some activities called for more flexibility to respond to learning during implementation. Fourth, given the long term nature of land management approaches to sustainability, more attention to intermediate outcomes, such as early zero tillage yield trends and tree seedling survival rates, would have better enabled an assessment of the longer term expectations.

4. Achievement of Objectives (Efficacy) :

Objective1. The extent to which the project arrested and reversed land degradation processes in riparian ecosystems and adjacent agro-ecosystems by increasing on-the-ground investments to encourage sustainable land management. Modest.


(i) 150 communities were reached with investment and training support and 150 micro-catchments were at least partially treated (100% of target).

(ii) 317 farms adopted sustainable land management ( 35% of the target of 900).

(iii) Sustainable land management was adopted on 32,868 ha, slightly exceeding the target. The reduction in number of farmers compared to the target suggests a larger area of adoption per farmer than originally expected. The land area figure of 38,868 ha may be somewhat misleading since it appears to cover all the land a farmer committed to managing in a sustainable manner, not necessarily how much each actually managed sustainably during the project period. The ICR reports (page15) that, in addition to the direct project achievements, the adoption of sustainable land management practices went beyond the targeted areas through individual farmers, producer cooperatives, associations, NGOs and agro-industrial interests. Thus, the reduced number of farmers was at least partly compensated for by an additional larger number outside the project. Some of this may be attributable to the project through the demonstration effect.

(iv) Riparian forest rehabilitation was adopted on 401 ha (27% of the target of 1,500 ha). This output target was revised downwards to 500 ha during implementation. About 7,200 people were trained in restoration techniques and 255,000 were reached by newsletters, radio programs, and workshops.

(v) There was an awareness target of 48,000 people, and a capacity building target of 4,800 people. Output achievement is not reported. The relevant project component was delayed and the State Secretariat for Environment's final report (ICR, page 10) notes "duplication, fragmentation and redundancy" in many sub-components. The delays in implementing the training in schools meant that it was too late to achieve a mass impact. 42% of the funds allocated for the component was disbursed.

It should be noted in assessing target shortfalls that the direction of the interventions had potential to impact rates of degradation and that the original PAD Results Framework had signaled in advance the uncertainties, and the consequent possibility of shortfalls, by setting project year milestones by which to re-evaluate targets. For example, it was intended to re-evaluate the number of micro-catchments by Project Year 2 had the target lagged the original 150 (it did not). There was an intent to re-evaluate the number of farmers participating by Project Year 2 if, by then, less than 50% of the target had been achieved (this turned out to be the case). It was also the intention to re-evaluate the sustainable land management area target by Project Year 4 (not necessary).

The project did not achieve the unrealistically high seedling production targets. However, statewide seedling output over the project period increased substantially due to increased private production, a positive development for institutional sustainability. The ICR (page 16) states that some of this private sector activity was attributable to project-supported demonstrations and dissemination. Although this is plausible, no evidence is offered.


There is, as yet, no evidence that degradation was arrested or reversed on any scale or that significant carbon was sequestered.

The ICR points to evidence from the (earlier, but similar) Land Management III project that the outputs achieved could be expected to lead to reversal of degradation once land management treatments mature. For example, it is known that the benefits of zero tillage typically take about five years to start to become apparent (with declines in yield sometimes experienced in the initial years). Forest restoration with seedlings and management takes more years than the project lifetime to yield measurable outcomes, particularly in terms of erosion reduction or hydrology enhancement. The ICR (page 15) points out that, even at the modest scale of the micro-catchments treated under the project, there would have been some possibility of finding some early measurable impact if all restored areas had been concentrated in the same micro-catchment. However, they were, in fact, dispersed across river basins for the purpose of testing restoration under different conditions. The ICR does not cite intermediate indicators like seedling survival rates, initial soil loss measurements, or initial yields from zero tillage, which would have offered some evidence on which to base expectations. Data on yield changes in another project (stated to be comparable) indicated a 23% increase in cotton yields over 5 years and a 100% increase in dairy productivity.

Other contributing factors in assessing future expectations include: (i) the legal framework (discussed under Objective 2); (ii) studies which provided an array of lessons for the future (most of which were not funded by the project; and, (iii) the testing of ten alternative models that demonstrated, inter alia, the high cost of some of the more intensive restoration activities, a useful finding for longer-term efficiency. The testing of several alternative treatments was a significant contribution.

The raising of awareness and education programs was expected to contribute to the above objective although the numbers trained appear to be modest in relation to the needs of the State.

Objective 2. The extent to which the project arrested and reversed land degradation processes in riparian ecosystems and adjacent agro-ecosystems by strengthening the policy, regulatory, economic, and institutional incentive framework to encourage sustainable land management. Modest.


(i) A procedure for Payment for Environmental Services (PES) was supported by the introduction of legislation approved in 2009 and instituted within the State Climate Change Law. By the end of the project, there were 21 pilot municipalities testing the PES system with project support. Additional programs were planned (ICR page 15).

(ii) A Riparian Areas Communication tool was established through which farmers voluntarily register riparian areas on private properties and undertake that, at a minimum, they will leave them unutilized to enable regeneration. This now covers 400,000 ha, far more than the area covered directly by the project investment.

(iii) A Riparian Forest Restoration Program system was designed and was instituted by a State Resolution. An implementation plan was developed and applied. The final evaluation is reported by the ICR as finding that the project spawned an impressive body of studies. This included work related to the operation and management of the project; regulations on new methodologies; financial incentives facilitating and requiring collaboration between government, municipalities and civil society on forest restoration; and improvements to existing laws and regulations to establish riparian forest issues within a legal framework.

(iv) The establishment of new institutional partnerships supporting the integrated conservation, socioeconomic and sector goals. Key partner agencies in investments included BNDES (National Development Bank), FEHIDRO (the State Water Resources Fund), and PETROBRAS (the semi-public Brazilian multi-national energy corporation).


This objective was stated largely in output terms. The ICR (page 18) states that the project had a marked and durable impact on institutional capacity, and that a collaborative dialogue evolved between the agriculture and environment sectors. It also states that a new management culture evolved in the State Environment Secretariat, including greater control over the timing of activities and the quality of projects’ physical and financial management. However, little evidence is presented in the ICR to support these statements or about the impact of the enhanced frameworks on practices and processes. The contribution of capacity building to arresting land degradation is covered under the discussion of Objective 1. The project made an important contribution to the development of the legal amendment to the existing Climate Change Law that laid the basis for Payments for Environmental Services, previously not permitted, however, actual payments were only tested on a small scale during the life of the project.

5. Efficiency:

Efficiency is rated modest due to limited evidence, lack of data on operation and maintenance at the farm level, and the reduced scale of achievement measured against the original investment-related indicators.

Evidence that project investment was least cost is limited. The ICR offers no industry standard comparator costs, cost or time overruns or underruns, or procurement efficiency evidence. Qualitatively, on the negative side, the substantial reduction in the original targets for forest restoration suggests that overheads on that activity are likely to have been significantly higher than originally anticipated. There is no evidence on maintenance costs. The complexity of the institutional relationships suggests substantial coordination costs. The unrealistic phasing for the findings of tested technologies to influence subsequent project activities suggests some costs in terms of reduced or delayed benefits. Restoration management plan preparation was found to be costly suggesting that they would be difficult to replicate on a larger scale. On the more positive side, additional financing came from sources outside the project offering some leveraging. There were synergies exploited between two ongoing projects, this project and the LM III project suggesting some cost savings. There were cost reduction opportunities found by the restoration testing with potential savings for the longer term.

There is very little evidence on the efficiency of specific land management interventions and none on that of the project as a whole . No ex post internal rate of return (IRR) was estimated, although in the PAD there was a cost effectiveness analysis and a partial IRR. According to the ICR (page 45), the PAD IRR, although stated as being for the whole project, in fact only covered the sustainable land management component. It appears to have been taken from the Land Management III PAD .

As noted by the ICR, (footnote, page 17), "It was not possible to provide an IRR as benefits were not measured, only costs, as is typical in GEF projects". According to the ICR, proxy measurements from the Land Managament III project carried out by the Food and Agriculture Organization (FAO) on similar types of intervention suggest satisfactory rates of return, although these appear to refer mainly to agricultural activities. Annex 3 of the ICR focuses largely on the reduction in restoration costs that the testing of the project achieved. It notes that the total savings from the lowest cost restoration option, assuming a target of one million hectares, would eventually be about US$2 - 3 billion over the proposed program lifetime. However, in terms of attribution of such savings, it is not easy to see how the lower-cost options would not have evolved, perhaps more gradually, without the project. "In-filling" as in this case within partially degraded forest areas is a widely used cost-reducing practice, although local conditions call for local adaptation.

The ICR (page. 47) notes that attempts to use farmers for carrying out restoration work at possibly even lower costs were prevented by the legal inability of the State Secretariat of Environment to make direct payments to landholders. New Payment for Environmental Services legislation now allows this, although it came too late to benefit the project interventions. The extent to which this would further reduce costs is not yet clear.

There was only one study (Cabral 2010) (cited in the ICR page 47) that gives some indications of a benefit stream. The study estimated the annual savings through reduction in the cost of water treatment and the dredging of reservoirs in representative micro-watersheds in São Paulo. The savings appear to be substantial, perhaps as high as US$0.5 million annually. However, this benefit stream has not been paired in the efficiency analysis with any cost stream so it is difficult to know what it means in terms of its contribution to overall project economic efficiency.

FAO analyses of similar forest restoration interventions under the Land Management III Project are reported to have found that the activities were economically and financially profitable; the income of farmers who adopted them increased significantly with higher productivity in both cotton and dairying. However, these are not findings from this project.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:

Rate Available?
Point Value
ICR estimate:

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Efficacy is modest: the attainment of outputs against targets was variable, falling short in some important areas. There is, as yet, no evidence that degradation was arrested or reversed on any scale or that there was significant carbon sequestration. It is also too early to assess the outcomes of the regulatory and institutional reforms supported. Efficiency is modest given the lack of attributable evidence on cost effectiveness. While the Global Environmental Objectives were substantially relevant, the scale and timing of project activities were incongruous with the development objectives as stated, and there were other weaknesses in design relevance, which is assessed as negligible.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

A new Sustainable Rural Development and Access to Markets Project approved in May 2010 is now effective and is expected to continue support for the improvement of environmental sustainability. It will be applying lessons from this project. It will scale up the use of no till agriculture, green fertilization, terracing, fencing of springs and riparian margins, gully stabilization and rotational pasture management. There remains a question about how far the Borrower's budget along with any new donor support in the longer term can carry eco-system restoration towards the ambitious project GEO.

A significant weakness in terms of information related to sustainability is that no data were captured to indicate the extent to which beneficiaries were maintaining improvements introduced on their land through the project activities. However, there is evidence that farmers and groups are expanding the use of some of the technological improvements demonstrated.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
On the positive side, the project was designed to address the relevant area of land management, supporting some appropriate legal, policy, and institutional changes. According to the ICR (page. 20), confirmed in discussion with the project team, there was a good skills mix: the preparation team included a Senior Environmental Specialist, a Senior Anthropologist, and a Social Development Specialist.

However, on the negative side, there were significant weaknesses: (i) there was an incongruity between the objectives and the much more modest outcome indicators so that design did not reflect the objectives; (ii) the M&E framework was flawed; (iii) the project timetable was too short for most intended outcomes to become evident; and, (iv) the project was over-complex, calling for an unrealistically demanding level of coordination and skills.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
There were four TTLs in five years which made continuity challenging. But according to the project team, the Region was able largely to mitigate this with good information retention and coordination. There were no significant issues with fiduciary or safeguard aspects. The ICR points (p. 21) to good technical collaboration on the Payment for Environmental Services activities, an area where skills and experience are in short supply globally. The Mid-term Review tackled relevant emerging issues. However, M&E weaknesses were not sufficiently addressed and rectified including incompatibility between the project's M&E system and that of the extension system.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The ICR (p. 21) reports that the Government of Sao Paulo supported the project, including the legal and regulatory framework to support forest restoration and sustainable land management on a larger scale than hitherto. The ICR points out that this was a significant political achievement given the status of the State as an agricultural powerhouse with potential political costs to sweeping conservation reforms impacting land use on potentially productive land.

The Secretariat of the Environment is reported to have been pro-active in the promotion of important policy reforms. The ICR notes that the government has already scaled up the project approach through a number of new or improved programs using public funds and incentives.

However, there were weaknesses in coordination and capacity, partly arising from project complexity.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:
The implementing agencies were the State Secretariat of Environment (SMA) and the State Secretariat of Agriculture and Supply including the State Rural Extension Company. The ICR notes that this was a pioneering effort in institutional collaboration and the associated dialogue was important for longer-term conservation.

There were a number of initial problems due to the complexity of the project and weak capacity. These were partly addressed by the restructuring in 2008/2009. Local capacity was supported by ensuring that the project management and coordination was mainstreamed within SMA rather than through an isolated temporary PMU. The State Rural Extension Company was able to apply its experience to support farmers in participating in the project. However there were issues in coordination particularly on M&E. Overall, M&E remained weak.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The PAD (Annex 3) contained a fairly detailed plan for M&E and a substantial set of detailed, predominantly output level, indicators. A detailed Project Monitoring and Evaluation Plan was included in the Operating Manual. However, there were a number of weaknesses in M&E design. It had been believed essential to integrate the M&E with the State Rural Extension Company system but this proved incompatible with the project complexity and the management structure. This significantly weakened the ability to respond to issues as they arose.

The PMU was to prepare semiannual reports to be made available to project executors at all levels and also to the River Basin Committees. The specification of the proposed evaluation methodology drew from the experience of the precursor Land Management III project. There was to be an initial diagnostic study with the participation of the community, and then follow-up to assess a number of indicators including social organization, land structure evolution, soil changes, surface and groundwater quality, and flora and fauna changes. It was proposed that an independent external consulting firm would evaluate the results obtained from impact monitoring against the project objectives. As noted by the ICR, it was an ambitious design.

b. M&E Implementation:
The project MIS was duly established. The project produced a Mid-Term Review, a final report, an environmental perception study and a final evaluation. PES practitioners were brought together to share lessons from other parts of the country. A book on the PES experience is to be published. The ICR reports (p.12) an impressive body of research studies and guidance notes, monitoring methodologies and policy formulation (ICR Annex 9). However, the weaknesses noted above carried over into implementation, particularly the design incompatibility with the extension M&E system.

a. M&E Utilization:
While the ICR offered less information about the levels and extent of utilization, it is clear that adjustments were made during implementation and that these drew at least partly from the documented project learning experience.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
Safeguards triggered by this Category B project were: OP4.01 on Environmental Assessment, OP4.09 on Pest Management, and OP 4.37 on forests. No significant safeguard issues were reported. "Supervision found that the identification and implementation of activities on the ground followed recommended practices consistent with the project's Environmental Management Plan, reduced the need for pesticides and fertilizers, avoided further deforestation of riparian/other areas within the 15 pilot micro-catchments and promoted re-planting of these same areas" (ICR, p. 12).

b. Fiduciary Compliance:
Financial management through the project period was rated satisfactory or moderately satisfactory. Audit reports were generally on time although there were two that were delayed. While there were no serious accountability or control issues raised by auditors, there were some auditors' qualifications which the ICR reports (p. 13) as having been addressed. Procurement was rated satisfactory by supervision missions throughout. According to the ICR (p. 13), no significant procurement issues arose.

c. Unintended Impacts (positive or negative):
The ICR reports (p. 15) that there was a strong multiplier effect from the project in that numerous micro-catchment communities, NGOs and municipal governments were developing forest restoration projects using other public and private funding. Also, the state's Multiyear Development Plan of 2008 to 2011 institutionalized riparian forest restoration, and this has continued. The Project stimulated the Riparian Areas Communication tool through which farmers voluntarily register riparian areas on private properties and undertake that, at a minimum, they will leave them unutilized to enable regeneration. This now covers 400,000 ha, far more than the area covered directly by the project investment.

d. Other:

12. Ratings:

IEG Review
Reason for Disagreement/Comments
Moderately Unsatisfactory
Efficacy is modest: the attainment of outputs against targets was variable, falling short in some important areas. There is, as yet, no evidence that degradation was arrested or reversed on any scale or that there was significant carbon sequestration. It is also too early to assess the outcomes of the regulatory and institutional reforms supported. Efficiency is modest given the lack of attributable evidence on cost effectiveness. While the Global Environmental Objectives were substantially relevant, the scale and timing of project activities were incongruous with the development objectives as stated, and there were other weaknesses in design relevance, which is assessed as negligible.  
Risk to Development Outcome:
Negligible to Low
There is no evidence on O&M of farm investments and therefore uncertainty about sustainability. There are some uncertainties about the state budget capacity to ensure continued financing. 
Bank Performance:
Moderately Satisfactory
Moderately Unsatisfactory
There were significant shortcomings in Quality at Entry due to a disconnect between objectives and design. 
Borrower Performance:
Moderately Satisfactory
There were weaknesses in coordination and capacity, partly arising from project complexity. M&E remained weak. 
Quality of ICR:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR offers a number of useful lessons, two of which (lessons 2 and 3) are taken from the ICR with some adaptation of language. Lesson 1 is a finding of this Review, although also reflected partly in one of the ICR lessons.

1. Consistency between the objectives statements and the scale of investment and associated indicators is essential to avoid confusing incongruities for all project participants between investment strategy and project intent.

2. In an environment-focused project with many institutional players, mechanisms should be designed during preparation and appraisal to formalize institutional partnerships and to integrate the different sectoral efforts for environmental conservation. These mechanisms need to be agreed among the partners and tailored to their own bureaucratic, managerial and operational structures.

3. In a project with many institutional players and with an experimental focus to develop the rules of the game and the processes, it is particularly important in designing M&E to get consensus on the coordination on M&E between institutions and sectors within government in order to ensure the alignment of objectives and targets, and to achieve the essential flow of M&E data between entities.

14. Assessment Recommended?

To verify the ratings and document lessons learned.

15. Comments on Quality of ICR:

The ICR is of generally sound quality. It presents the experience of the project thoroughly and has a number of useful lessons. It correctly notes the excessive ambition of the GEO.

a. Quality of ICR Rating: Satisfactory

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions