Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Punjab Forest Sector Dev.


  
1. Project Data:   
ES Date Posted:
06/27/2002   
PROJ ID:
P010481
Appraisal
Actual
Project Name:
Punjab Forest Sector Dev.
Project Costs(US $M)
 33.75  17.61
Country:
Pakistan
Loan/Credit (US $M)
 24.87  12.81
Sector, Major Sect.:
Forestry,
Agriculture
Cofinancing (US $M)
   
L/C Number:
C2747      
   
Board Approval (FY)
  95
Partners involved
 
Closing Date
12/31/2001 12/31/2001
         
Prepared by: Reviewed by: Group Manager: Group:  
Ridley Nelson
Nalini B. Kumar Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
The overall objective stated in the SAR was "to improve the performance of the forestry sector in Punjab". More specific objectives included: (i) (establish) conditions that would encourage a broadening of the productive base of wood from farm land; (ii) (provide) demonstrations of scrub forests, rangeland and plantations rehabilitation and management that maintain the availability of common benefits and forestall deterioration of resources; and (iii) strengthen local, public, and community institutions managing the sector in Punjab. The project was a PTI (Program of Targeted Interventions).

b. Components
The project components were: (i) Institutional Support (US$2.19 million) - including a human resources study to guide institutional reform, an Administrative Unit for implementation, and strengthening the existing Monitoring and Evaluation Unit; (ii) Farm Forestry (US$21.95 million) - including technology transfer to expand private sector and farmer participation, improving the supply of superior planting material, and adaptive research; (iii) Social Range and Scrub Forest Management (US$3.52 million) - including rehabilitation of about 21,000 hectares of range and scrub forests, introduction of participatory planning and management systems through community organizations, testing mechanisms of cost recovery, providing training to Forest Department officials and farmers in participatory approaches, and supporting applied research on fodder trees and shrubs; and, (iv) Irrigated Timber Plantations (ITPs) (US$2.88 million) - including preparation of a biodiversity inventory, completion of a baseline study of ITP's, preparation of management plans for priority plantations, and implementing a pilot management and rehabilitation plan for three representative plantations. There was no major revision or cancellation of components during implementation.

c. Comments on Project Cost, Financing and Dates
Due largely to local currency devaluation but also the failure to complete one component, there was a substantial cost underrun leading to cancellation of about 35 percent of the original credit amount.


3. Achievement of Relevant Objectives:

Given the weaknesses in M&E efficacy is difficult to assess. Objective (i), which was to establish conditions to broaden the productive base, appears to have been largely achieved. The main institutional and policy prerequisites for achieving this objective are now in place. Three key policy changes were achieved in association with the project: the subsidy on tree seedlings was removed (although free seedlings are still provided to government departments), investment in public nurseries was reduced to create incentives for private production (although there are still some FD nurseries), and charges on the movement of wood were removed - which had been a disincentive in the past to planting trees on farmlands. The achievements in the Farm Forestry component - by far the largest component - appear to have been satisfactory with an impressive 272 million seedlings being planted on farms, although measured survival rates and growth rates are not clear. The borrower notes that, with this project, farm forestry production in Punjab has now reached saturation point with marketing now being the main future issue. Based on an impact study income from trees was estimated to have increased by about 20% in real terms. Objective (ii), which was to provide demonstrations of rehabilitation and management through community-based processes appears to have largely failed; there are serious questions about sustainability. In the absence of support from District Governments the borrower expresses doubts about sustainability of these community organizations and the Pujab FD uses the words "have become dysfunctional". Objective (iii), related to institutional strengthening, was mostly achieved but with significant shortcomings. A farm forestry and extension wing in the Forest Department was created, a corp of "greenmen", consisting of both retrained forest guards and new recruits was established to do extension work. While key professional staff for the Monitoring and Evaluation Unit were provided, it never functioned adequately. A staff training package was implemented. The Punjab Forest Research Institute was strengthened. However, neither the Human Resource Development Study nor the international training program were fully implemented. Most importantly, the very important marketing intelligence system was never established.

4. Significant Outcomes/Impacts:

The policy reforms achieved related to tree seedling subsidy and charges on the movement of wood were important. The start of a shift from a "command and control" Forest Department to consultation with community members is a significant change of direction albeit on a small-scale and with serious concerns about sustainability.There is limited data on outcomes and impacts in terms of forest productivity. Survival rates of plantations and management of treated rangelands are reported to have been generally good with one estimate under the range/scrub component of 80 percent seedling survival. For the range/scrub component an impact study by a reputable research institute was undertaken which reported a 20% to 30% income increase due to livestock from improved fodder. By project closing there were about 800 private nurseries which had produced by then about 270 million seedlings - somewhat exceeding the appraisal target - the equivalent of about 150,000 hectares of plantation. Farm forestry extension is reported to have been strengthened, working through contact farmers and using a range of audiovisual support. A tree seed supply system was developed - important for the future - and there were a number of achievements in research including training in tissue culture.

5. Significant Shortcomings (include non-compliance with safeguard features):

There was limited consultation with stakeholders during preparation. The institutional development impact is rated by the ICR as modest mainly because the financial management component, which did not receive sufficient attention during preparation and appraisal, did not work well. The planned marketing intelligence system was not established by the closing date. The M&E Unit results were poor due to insufficient management support, insufficient staff and disinterest in the use of MIS software by the field units.The problem arose partly because there was no M&E design at the outset - this was left to be done in the first year of the project. There was also a rapid turnover of project staff, in fact the project had 11 Directors over the 6 year project period which, among other things, negatively affected flow of funds. The irrigated timber plantations component (smaller than the farm forestry component) is rated, by the ICR, as unsatisfactory because the management and rehabilitation plans were not implemented. There are some concerns that many of the new private nursery operators will not survive unless they scale up their business, diversify, and respond better to changing demand. About 100 community organizations managing about 250 hectares each of rangeland and scrub forest were initiated. However, only 50 were intensively supervised of which 32 were expected by the ICR to be sustainable but the Punjab FD calls them now dysfunctional. The project was negatively affected by drought over the final two years and by a disease of one tree species which caused a glut of dead wood on the market. More generally, pricing problems were also impacted by the failure to implement the marketing information system.There were problems with delayed release of funds from government.
6. Ratings:ICROED ReviewReason for Disagreement/Comments
Outcome:
SatisfactoryModerately SatisfactoryThe project is expected to achieve most of its major objectives but with significant shortcomings including: failure to establish a marketing intelligence system; suggestions by the Punjab FD that supply may have reached saturation point - disputed by the region (there are linkages between the last two points); questions about sustainability and therefore efficiency of the range and scrub community organizations component - Punjab FD call them dysfunctional; and the failure of the irrigated component. In retaining a satisfactory - albeit moderately - rating we weight highly the important policy shifts.
Institutional Dev.:
ModestModest
Sustainability:
LikelyLikelyThis rating is not without questions. Sustainability is defined as the resilience to risk and, notwithstanding significant tree-planting achievements, there are uncertainties about the resilience of this project in the future. Policy is certainly promising but there is a need for follow-up actions and sustained public resources. There are indications in the Punjab FD comments of a situation close to over-supply. The region disputes this. There are also some concerns about the sustainability of community organizations in the range component - a small component but a key element in demonstrating a participatory approach.
Bank Performance:
SatisfactorySatisfactory
Borrower Perf.:
SatisfactorySatisfactory
Quality of ICR:
Unsatisfactory

7. Lessons of Broad Applicablity:

The ICR offers many lessons - too many in fact - but a number confirm lessons from other projects. The following (with some adjustment in formulation from the ICR and one addition) are the most important: 1. An implementing institution managing a new project should be subject to a careful analysis of financial management capacity very early in preparation so that systems and skills can be developed and tested even before effectiveness. 2.Research components, due to the lag in impact, call for very early identification and completion of training needs so that the new skills can impact as early as possible. 3. Repeated interaction and good skills training backed up by positive stakeholder response can turn around authoritarian public service attitudes towards more participatory approaches quite quickly. 4. Establishing sustainable community-based resource management groups takes longer than a typical Bank project period. 5. Weakness in M&E is often due to the failure to design M&E during preparation, inappropriate institutional location of the M&E function, and lack of incentives to report.

8. Audit Recommended?  Yes

          Why?  There are a number of uncertainties about sustainability and there are potential lessons for community development approaches.

9. Comments on Quality of ICR:

While the ICR contains adequate information, and has a number of fully satisfactory elements, there are a number of weaknesses and on balance it therefore falls somewhat below a satisfactory rating: (i) by far the most important weakness is that there is a striking disconnect between the Punjab FD comments and the ICR and main borrower comments on several issues including product supply/demand, community organizations sustainability, district devolution issues and training. This raises questions which cry out for answers in the ICR. While the region disputes many of the Punjab FD comments and refers OED rather to the overall borrower comments, the fact remains that the Punjab FD comments are included, cannot be ignored because they come from the main implementing agency, and do cover potentially important issues. This disconnect should have been addressed in the ICR; (ii) this was a PTI (Program of Targeted Interventions) but performance against poverty objectives in the light of this classification is not evaluated in the ICR, although there are a few indications of poverty impact; (iii) one would have hoped to see more analysis on projected product demand and prices given that much of the production is still in the future, that the Punjab FD raises questions about supply/demand (using the word saturation), and that the ERR appears fairly sensitive to a fall in output price. Main lessons could have been fewer and more focussed but a number are well formulated.

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions