Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Third Forestry Development Project


  
1. Project Data:   
ES Date Posted:
06/27/2003   
PROJ ID:
P009582
Appraisal
Actual
Project Name:
Third Forestry Development Project
Project Costs(US $M)
 8.9  9.4
Country:
Bhutan
Loan/Credit (US $M)
 5.4  5.2
Sector, Major Sect.:
Forestry, General public administration sector,
Agriculture fishing and forestry; Law and justice and public administration
Cofinancing (US $M)
 2.7  3.8
L/C Number:
C2533      
   
Board Approval (FY)
  94
Partners involved
Swiss Agency for Development Cooperation 
Closing Date
06/30/2002 06/30/2002
         
Prepared by: Reviewed by: Group Manager: Group:  
Nalini B. Kumar
Ridley Nelson Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
The objective of the project was to support Bhutan's efforts to develop and implement an approach for sustainable protection, management and use of its forest resources in line with its national development priorities. To this end the project was to provide support for (a) carrying out sustainable forest management in selected government forest areas; (b) involving local people in the rehabilitation of degraded forest areas and their subsequent management; (c) integrating trees into farming systems; (d) introduction of a criteria based approach to afforestation and reforestation; and (e) strengthening the planning and implementation capacity of government organizations.

b. Components
The project had four components:

Forest management to introduce sustainable, scientific management of timber supply from natural forests in Eastern Bhutan as an alternative to the present ad-hoc, unsustainable utilization of forests;
Social Forestry to introduce a sustainable mechanism for village communities to manage forest resources near their villages, to establish community forest plantations near villages on highly degraded land and to introduce trees into private farming systems;
Afforestation/Reforestation to test alternative methods for effective regeneration on selected degraded government forest land;
Institutional Strengthening to strengthen the capacity of the Ministry of Agriculture and its Forest Division to develop Bhutan's forest sector policy, plan investments in the sector and manage sector and project activities effectively and efficiently.

At MTR it was agreed that SDC would provide additional funding for TA and accordingly the dollar amounts for three of the components were revised marginally.

c. Comments on Project Cost, Financing and Dates
The project was appraised in April 1992 and approved in July 1993. Mid term review (MTR) took place in December 1996. The implementation period was fairly long and the project closed in June 2002. As appraised total project costs were US $ 8.9 million of which the IDA contribution was to be US $ 5.4 million, SDC was to contribute US $ 2.7 million and the Government of Bhutan was to contribute US $ 0.8 million. Actual SDC contribution was US $ 3.8 million or nearly 40 percent of the total project cost. Actual IDA disbursement was US $ 5.2 million.


3. Achievement of Relevant Objectives:

The project was a complex operation with ambitious objectives which were difficult to achieve given the sub-national focus of the operation and the weak institutional capacity of the Borrower. The ICR realistically notes the advantages had the project been designed as a 'learning process' approach (ICR, para 4.1).

It is difficult for the Evaluation Summary (ES) to assess how far the objectives were actually achieved given the inadequate monitoring and evaluation (M&E) and poor data. The project had several shortcomings as noted in Section 5.The ES also has concerns about the environmental impact of the operation. As noted in the ICR the project included an early demonstration of reduced impact logging which requires a careful trade-off of commercial and practical considerations against complex ecological factors. Hence project interventions should have been accompanied by continuous environmental monitoring which could not be carried out because of the lack of capacity in the Borrower (ICR, para 7.5). However progress has now been made on this front and the government has piloted environmental monitoring procedures in the project area and adopted them for use at the national level.

The rate of return at completion is reported to be about 11 percent. The calculations are made on very conservative assumptions. However given the data limitations it may not be a reliable estimate.

4. Significant Outcomes/Impacts:

A significant body of applied research and practical experience is reported to have been developed on the management of Bhutan's broad leaf forests for future management of the sector; The ICR notes that findings of applied research contributed to improvement of felling practices and regeneration methods and improved methods were shared with private contractors. Given the lack of formal mechanism for evaluation training and poor M&E it is difficult to say how much learning actually took place. Poor environmental monitoring (section 3 above) makes assessment of impact at a broader level difficult.
  • Adoption of planned management to replace ad-hoc approaches to resource utilization that prevailed in the pre-project period;
  • Environmental standards and practices for the construction of logging roads are reported to have improved significantly though quality control and required maintenance remains uneven;
  • Paradigm shift in the government in favor of social forestry and development of capacity to undertake social forestry among government staff;
  • Inclusion of direct community participation in Forest Management Unit (FMU) planning and increased capacity for FMU planning;

5. Significant Shortcomings (include non-compliance with safeguard features):

There were several shortcomings in each of the components: Forest Management: The ICR rates this satisfactory with caveats. The project has been unable to fully resolve the primary threat to forest regeneration in the project area,i.e. cattle grazing; 16% of hardwood produced in FMUs supported by the project failed to find either a domestic or export market and was left decaying in depots. An adequate solution to the cause of the problem has been developed although the success of these measures are yet to be proven; only three of the five FMUs were in operation by the end of the project;
Social Forestry: ICR itself rates performance of this component as unsatisfactory. Key elements of the social forestry component fell short of expectations. Draft social forestry rules developed before the project became effective were not finally notified until the year 2000 significantly affecting implementation; delegation of authority for approving community forest management plans to the district level was never operationalized; continuing free distribution of seedlings could undermine the sustainability of private nurseries established during the project;
Afforestation/Reforestation: ICR itself rates performance of this component as unsatisfactory. Very little community participation, unrealistically high targets, very low survival rates in plantings;
Institutional Strengthening:Lack of a thorough upfront training need assessment negatively impacted outcome; civil service procedures and frequent staff transfers undermined capacity building efforts;

6. Ratings:ICROED ReviewReason for Disagreement/Comments
Outcome:
SatisfactoryModerately UnsatisfactoryWhile the ICR considers the project to be marginally satisfactory (ICR, para 4.1) the ES rates it as moderately unsatisfactory. OED rates outcome as moderately unsatisfactory when the project is expected to achieve its relevant objectives but with major shortcomings. The shortcomings noted in section 5 are major.
Institutional Dev.:
ModestModest
Sustainability:
LikelyNon-evaluableSustainability is rated as non-evaluable at this stage as some factors support a likely rating and others support an unlikely rating and there is not enough evidence to make a conclusive judgement.
Bank Performance:
UnsatisfactoryUnsatisfactory
Borrower Perf.:
SatisfactorySatisfactoryBorrower performance is rated satisfactory but only marginally so because of several factors, three of which are noted here: (i) several changes in sectoral policy and laws were implemented much later than envisaged with consequent negative impact on project implementation; (ii) general lack of institutional support and commitment from the central government to staff engaged in field implementation; (iii) civil service procedures caused delays in filling project positions or resulted in frequent staffing transfers undermining some of the capacity building measures undertaken by the project.
Quality of ICR:
Satisfactory

7. Lessons of Broad Applicablity:

The ICR notes several important project level lessons two of which are repeated here: (i) land for reforestation is often subject to other uses by local communities and therefore plantations have proven unsustainable in the absence of meaningful involvement of current users; (ii) the training policy needs to ensure that staff trained in a particular skill remain employed in that area of training and that all training is consistent with an established training program.
The ES adds the following:
(i) In projects with a clear poverty alleviation objective alongside an objective of regenerating forest cover, development objectives should be framed at least partly in terms of poverty alleviation. This should be done in a way that is monitorable and should be backed up in design by a monitoring and evaluation system capable of measuring poverty impact.
(ii) Policy issues such as seedling pricing and institutional issues like land tenure arrangements and linkages between forests and livestock grazing need to be identified and dealt with ex-ante. Solution to these issues requires commitment from various levels/departments of government. Hence, if they are not sorted out at the design stage, they may never get resolved and may dampen the implementation experience of the best of projects.
(iii) Development of an efficient monitoring and evaluation system is crucial to the establishment of a community based forest protection and management system.This is so not only for assessing outcomes and impacts but for making efficient mid course corrections, as and when required.
(iv) Staff continuity on the Bank side is as important as that on the side of the Borrower. In the Bhutan Third Forestry Development Project there were six Task Team Leaders over the life of the project with consequent negative impact on project performance.

8. Audit Recommended?  Yes

          Why?  For several reasons: (i) to verify the outcome, sustainability and institutional development impact; (ii) to realistically assess the poverty reduction potential of such interventions; (iii) to provide lessons of experience for similar interventions in other countries.

9. Comments on Quality of ICR:

The ICR is well written and provides a realistic account of the project. Given the data limitations the attempt at estimating the ERR is commendable.

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions