Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Nusa Tenggara Dev.


  
1. Project Data:   
ES Date Posted:
06/21/2004   
PROJ ID:
P004008
Appraisal
Actual
Project Name:
Nusa Tenggara Dev.
Project Costs(US $M)
 41.1  27.8
Country:
Indonesia
Loan/Credit (US $M)
 27.1  22.1
Sector, Major Sect.:
Agricultural extension and research, Irrigation & drainage, Animal production, Sub-national government administration, Roads & highways,
Agriculture fishing and forestry; Agriculture fishing and forestry; Agriculture fishing and forestry; Law and justice and public administration; Transportation
Cofinancing (US $M)
   
L/C Number:
L3984      
   
Board Approval (FY)
  96
Partners involved
 
Closing Date
09/30/2002 09/30/2003
         
Prepared by: Reviewed by: Group Manager: Group:  
Nalini B. Kumar
John R. Heath Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
The project objectives were (i) to raise smallholder incomes, (ii) strengthen local-level institutions and (iii) foster broad-based participation at the grassroots level. The provinces of West and East Nusa Tenggara were chosen because of their high incidence of poverty. Though the ICR claims that the original objectives remained unchanged throughout the project's life, yet at the time of the restructuring there was an implicit change in objectives as the project focus changed from general agriculture development to community-managed rural credit. However the evaluation is done against the original objectives as the restructuring was prompted by shortcomings in project design.

b. Components
The project had three components (with appraisal estimates and actual costs in parenthesis). (i) Establishment of the Naibonat Assessment Institute for Agriculture Technology (appraisal US$5.74 m, actual US$2.28 m). The component was designed to up-grade agricultural research and facilitate technology transfer to farmers.

(ii) Agriculture-Based Area Development (appraisal US$30.73m, actual 19.01 m). In 14 participating districts the component was to support on-farm demonstrations of applied research, distribution of farm inputs, technical training, support for NGOs as facilitators to motivate and assist beneficiary participation in project activities, improvement of farm-to-market access roads and establishment of a competitive grant fund to finance local community-identified initiatives.
(iii) Strengthening Agricultural Support Services and Project Administration (appraisal US$4.97m, actual US$6.48m). This component was to strengthen local government extension facilities and animal health centers, train extension workers and village leaders and provided technical assistance for project implementation and monitoring.

The project was restructured in February 1999 at the time of the Mid-term Review (MTR) when its scope was reduced and major revisions were made to the three components. Component (i) was transferred to a specialized research project. Component (ii) and (iii) were revised to give emphasis to community-managed credit, institutional strengthening and NGO provision of technical services and facilitation.

c. Comments on Project Cost, Financing and Dates
The project was appraised in May 1995 and became effective in June 1996. The Asian economic crisis began in January 1997, six months after the loan became effective, and had a major adverse impact on project implementation because of the devaluation of the Indonesian Rupiah. It led to the government request for cancellation of US$4.9m of the US$27m loan to reduce its foreign debt. It also affected the government's ability to provide adequate counterpart funding, particularly for recurrent costs. At appraisal, government contribution was expected to be US $13.97m. Actual government contribution was only US$ 5.68m.


3. Achievement of Relevant Objectives:

The achievement on the three objectives was as follows:
(i) Partially Achieved. The restructured project was able to raise smallholder incomes. A beneficiary impact survey carried out at the time of project closing found that investments financed by project household credit resulted in an increase in family income between 14 and 91 percent compared to the control group. The region in its comments clarifies that incomes went up by an average of 57 percent (weighted by type and distribution of loans). However, it is not clear that this was achieved in an efficient manner as an economic rate of return for the project based on increase in beneficiary incomes as captured by the beneficiary impact study was not calculated. The region subsequently clarifies that such analysis was not done because it was not feasible. The ICR provides evidence to show that had the policy environment for smallholder credit been favorable and the implementation challenges (several of them described in section 5) appropriately dealt with by the Borrower a much more favorable outcome could have been expected from this phase. The on-farm demonstrations accompanied by grants of inputs and the provision of cattle in the pre restructuring period had little impact on productivity and the village infrastructure established at that time did not lead to measurable increase in market access.

(ii) Partially Achieved (based on clarification and evidence provided by the region). Community managed micro-finance entities were established and trained to deliver loans to individuals and groups. However the necessary upward linkages with financial institutions was not made.

(iii) Partially Achieved (based on clarification and evidence provided by the region) .The project was instrumental in starting a process of participation in the villages. The ICR however notes that the restructured project was constrained by the inappropriate selection of participating villages. The region does not provide a rationale as to why the participating villages could not be changed at the time of the restructuring even though they had been earlier 'preselected by a process not related to their economic potential and willingness to cooperate' (ICR page 5.4.6)

4. Significant Outcomes/Impacts:

Collaborative links were established with universities and extension agencies for on-farm demonstrations and pilot trials of research results;
  • A total of 166,000 beneficiary families in 477 villages received credit which helped them increase their incomes. The largest loan purpose was for diversified non-farm activities;
  • Though the quality of the loan portfolio in East Nusa Tenggara was lower than that in West Nusa Tenggara, overall significant progress was made in securing repayments from beneficiaries.

5. Significant Shortcomings (include non-compliance with safeguard features):

Adequate attention was not given to several issues critical to project implementation in the pre restructuring period: timing of delivery of inputs for planting, availability of qualified staff, arrangements for technical supervision of works, local procurement, audits of works, budget resources, means of transportation for staff to get to the field. Several of them, like persistent budget approval and release problems, transport constraints, lack of adequate staff skills, continued to challenge implementation in the post restructuring period.
  • Critical lessons from the earlier project in the area (e.g. the ICR of the earlier project had noted the doubtful rationale for district-based IADPs, yet the current project was designed on the same basis (ICR para 7.1.1) were not learnt and mistakes were repeated leading to considerable wastage of time and resources;
  • In the post restructuring period, adequate linkage of the community managed micro finance units with other financial institutions were not made with negative implications for sustainability;
  • The lack of a defined micro-finance policy to provide sustainable, savings-led financial services to the rural poor negatively affected project performance as also the absence of a suitable write-off and re-scheduling policy once the loan was restructured;
  • Even in the post restructuring period, there was a poor fit between services required from TA and those available. The firms and personnel contracted in response to the original project design were retained far too long after the restructuring;
  • Though relevant data was collected through M&E, it was not effectively utilized to contribute to effective project management.
  • 6. Ratings:ICROED ReviewReason for Disagreement/Comments
    Outcome:
    UnsatisfactoryModerately UnsatisfactoryOn the basis of additional information provided by the region. OED rates a project as moderately unsatisfactory when it is expected to achieve its major relevant objectives with major shortcomings. The shortcomings noted under section 5 are major.
    Institutional Dev.:
    ModestModest
    Sustainability:
    UnlikelyUnlikely
    Bank Performance:
    SatisfactoryUnsatisfactoryQuality at entry was highly unsatisfactory. Quality of supervision was also wanting. Even after the project was restructured and its scope considerably reduced there was confusion over the project activities, inappropriate selection of participating villages and poor fit between TA available and needed. M&E was not utilized effectively. Given the difficult experience with other rural development projects in Indonesia, the Bank should have provided greater management oversight to design and supervision of this intervention.
    Borrower Perf.:
    SatisfactoryUnsatisfactoryThe operational manual was not produced until after the MTR. Involvement of too many agencies and lack of coordination between them was a major issue in the pre restructuring period. After the restructuring, the budget cycle, persistent shortfalls in budget and the political process continued to negatively impact project performance. The late release of the budget each year even after the restructuring did not allow project staff to adequately carry out local supervision and affected the timely contracting of NGO-supplied Community development Facilitators. The poor fit between government's general policy towards micro-finance and the project created confusion especially when the subsidized programs continued in the same or adjacent villages. There were procurement irregularities and delays and poor procurement management on the part of provincial authorities.Firms and personnel contracted for TA in response to the original design were retained too long after the restructuring resulting in a poor fit between services provided and needed. Selection, supervision and management of Community Development Facilitators was often unsatisfactory as also the selection of activities and participating villages.
    Quality of ICR:
    Satisfactory

    7. Lessons of Broad Applicablity:

    Based on the ICR the key lessons are:
    1. Close attention to ensuring good quality at entry can help avoid severe implementation problems down the road.
    2. A multi sectoral project can be complex and challenging to implement. Designing of such projects requires a thorough assessment of the institutional capacity in the country.
    3. The building of adequate community participation requires time, resources and explicit attention to incentives for the stakeholders--factors that need to be explicitly recognized and built into a project. Setting up grass root organizations is just the first step. More time and resources are needed to strengthen and consolidate gains so that their sustainability over the long run is assured.
    4. Merely collecting data through a monitoring and evaluation system is not enough. There is also a need to build a plan to ensure that observations are evaluated and feedback is effectively utilized. Only then can M&E be effective.
    5. Successful implementation of a rural credit intervention requires adequate attention to the broader policy environment for micro-finance in the country.

    8. Audit Recommended?  Yes

              Why?  To adequately assess Bank and Borrower performance and to verify the outcome and sustainability ratings (including validation of beneficiary impact survey results).

    9. Comments on Quality of ICR:

    The ICR is satisfactory but for four significant shortcomings:(i) the additional information on achievement of project objectives provided by the region in its comments should have been included in the ICR; (ii) there appears to be a contradiction: the ICR para 4.1.1 claims that after restructuring implementation improved substantially, however other parts of the report show that implementation continued to face several critical challenges; (iii) the ICR should have provided a rationale as to why the inappropriate participating villages were not changed at the time of the restructuring; (iv) the reasons why the economic rate of return for the project was not recalculated based on increase in beneficiary incomes as captured by the beneficiary impact study provided in the region's comments should have been included in the ICR.

    © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions