Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Afghanistan: Horticulture And Livestock Productivity Project


  
1. Project Data:   
ICR Review Date Posted:
03/12/2014   
Country:
Afghanistan
PROJ ID:
P098256
Appraisal
Actual
Project Name:
Afghanistan: Horticulture And Livestock Productivity Project
Project Costs(US $M)
 69.30  67.53
L/C Number:
CH226
Loan/Credit (US $M)
 20.00  21.48
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
 49.30  46.06
Cofinanciers:
Afghanistan Reconstruction Trust Fund
Board Approval Date
  05/25/2006
 
 
Closing Date
12/31/2009 12/31/2011
Sector(s):
Central government administration (32%), Crops (23%), Animal production (19%), Agro-industry marketing and trade (16%), Agricultural extension and research (10%)
Theme(s):
Other rural development (25% - P) Conflict prevention and post-conflict reconstruction (25% - P) Export development and competitiveness (24% - P) Gender (13% - S) Rural services and infrastructure (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Ebru Karamete
Ridley Nelson Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
The project was designed as a 3 year emergency operation. The Project Development Objective, stated in the Project Appraisal Document (p.6), is to " stimulate marketable output of perennial horticulture and livestock in focus areas by: (i) improving the incentives framework for private investments; and (ii) strengthening institutional capacity in agriculture" .


The Financing Agreement statement of Project Development Objective is (p. 4): "to assist the Recipient in stimulating marketable output of perennial horticulture and livestock by improving the incentives framework for private investments and strengthening institutional capacity in the agricultural sector".

The two statements are effectively identical but as per IEG’s current practice, this Review’s assessment is based upon the formulation of the project objective as in the Financing Agreement.

The project was formally restructured with the following revised objectives: "to assist producer households in adopting improved practices so as to increase horticulture and livestock productivity and production in Focus Areas" (p. 3 of Amendment to the Financing Agreement).

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 12/09/2009

c. Components:
1. Increasing Marketable Output of Perennial Horticulture (Appraisal Estimate (US$ 23.7 million, Revised Estimate US$ 18.89 million, Additional Financing Estimate US$ 4.36 million; Actual US$ 27.56 million) - This component aimed to provide civil works, goods, services, and grants to: (i) rehabilitate existing orchards and establish perennial tree crop cultivations; (ii) establish a Horticulture Development Council of Afghanistan and develop Ministry of Agriculture and Irrigation capacity for policy planning and supporting horticulture producers; and (iii) support pilot activities in private sector export clusters for green raisins and pomegranates (in collaboration with IFC).

2. Increasing Output and Productivity of Livestock (Appraisal Estimate US$20.2 million, Revised Estimate US$ 15.00 million, Additional Financing Estimate US$ 3.85 million, Actual US$ 18.19 million) - This component aimed to enable Afghan livestock owners to increase their access to and use of public and private livestock-related goods and services, to ensure the marketing of more and safer livestock and livestock products, and to improve the populace’s diet through better nutrition.

3. Capacity Building, Implementation, and Monitoring and Evaluation Support (Appraisal Estimate US$ 6.0 million, Revised Estimate US$ 18.49 million, Additional Financing Estimate US$ 6.79 million, Actual US$ 22.93 million) - This component aimed to provide goods, services, and incremental operating costs for developing human and physical capacity in policy formulation, programming development, financial management and procurement, supervising and monitoring, and evaluating impact.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs:
Total Project Costs increased from the original appraisal estimate of US$ 49.87 million to US$ 54.3 million. Also additional financing increased the project costs to US$ 70.07 million. At project closing, project costs were US$ 66.83 million and US$ 3.25 million was canceled.

Financing:
Financing initially comprised an IDA Grant of US$ 20 million, Japanese Social Development Fund Grant of US$ 10.0 million and IDA administered Afghanistan Reconstruction Trust Fund financing of US$ 20 million and beneficiary contribution of 1.06 million. JSD Financing did not eventuate. The reason was not provided by the ICR or the task team later on. Project restructuring increased financing under Afghanistan Reconstruction Trust Fund to US$ 34.3 million and additional financing added another $ 15 million under Afghanistan Reconstruction Trust Fund. The actual disbursements at the time of closing were US$ 46.05 million under Afghanistan Reconstruction Trust Fund and US$ 21.48 million under the IDA Grant. Beneficiary contribution was a negative figure at project closing (US$ -0.7 million). The team's response to why this occurred was: "Contributions were counted towards the revised financing estimate during the additional financing. Although contributions were collected, the Bank kept disbursing at a rate of 100% against expenditures for Goods, works, non-consulting services, consultants’ services, training and Operating Costs for the Project. Since the Ministry of Agriculture, Irrigation and Livestock ran out of time to use those funds in acceptable project expenditures before closing, it was asked to return the money to the Bank; resulting in negative number in the disbursement column and the lower Bank final disbursement when adjusted for this fact". This explanation was unclear: if beneficiaries did contribute but these funds were returned back due to not being used, then the beneficiary contribution should be zero, rather than a negative number.

Borrower Contribution:
The Borrower was not expected to make a contribution.

Dates:
With the restructuring on December 9th, 2009, the closing date of December 31, 2009 was extended for 2 years until the end of 2011 due to slow disbursement and not being able to achieve targets by the closing date. The additional financing which was approved in November 2011 to scale up activities also extended the closing date for another year until December 31, 2012.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Original Objectives:

Substantial
The project development objective of stimulating marketable output of perennial horticulture and livestock was relevant to country priorities and sector strategies but very ambitious for the three year timeframe and circumstances. Agriculture was key for overall economic growth and poverty reduction. During the time of appraisal it was central to the Afghan economy, contributing to 53 % of GDP and providing employment for 67 % of the labor force (PAD, p. 2). About about 80 % of the population lived in rural areas and incidence of poverty was high (ICR, p. 1). However, as a result of many years of conflict agriculture had suffered significantly resulting in decreasing yields and high value sub-sectors such as horticulture and livestock were neglected. These had been a substantial part of the economy and exports before conflict. Therefore helping to stimulate these sub-sectors sectors was essential for rural poverty reduction and economic development. In addition this would also provide a core element of a counter-narcotics strategy by fostering an alternative to the opium industry which had become an important income source, employing 17% of the population in rural areas (PAD, p. 3). Building State capacity to create an enabling environment that included the private sector was also essential for the sector to grow.

Agriculture with particular consideration to perennial horticulture, animal health, and food security was recognized in the Interim Afghanistan National Development Strategy and the associated 5-year Agriculture Sector Development Master Plan developed by the Ministry of Agriculture (2005). However, implementation of the strategy was impeded by the prolonged conflict that impacted the horticulture and livestock sectors, leaving the capacity of the government and the private sector not sufficient to assist the agricultural sector given the lack of basic infrastructure. Therefore, the Government requested World Bank assistance in developing policies and programs for the agriculture sector (ICR p. 1).

Although as a three-year emergency operation, the project was not part of the Country Assistance Strategy and was inserted later following an urgent request from Government, project development objectives were relevant to the World Bank's Interim Strategy (2012-2014) but optimistic. Under the "Inclusive Growth and Jobs" Pillar of the strategy, agriculture was identified as one of the critical sources of growth and employment.

Revised Objectives:
High
The revised project development objective, more realistic under the country circumstances and a clearer statement focusing more on adoption (assisting producer households in adopting improved practices ... to increase horticulture and livestock productivity and production in Focus Areas) was highly relevant to country priorities, strategies and World Bank strategies and programs.

b. Relevance of Design:
Original Design

Negligible
The original project design had a major shortcoming within the results framework: a broad and vague PDO that was very difficult to achieve in Afghanistan in only 3 years, and the results framework that did not meet the ambitious objectives. The objective of stimulating marketable output of perennial horticulture and livestock in focus areas was to be achieved through 3 components: (i) horticulture development that mainly aimed to rehabilitate existing orchards and establish new ones, and develop capacity of the MAIL; (ii) livestock development that tried to develop capacity of the General Department for Livestock Production and Development, as well as promoted private-public sector partnerships for the delivery of veterinary services, and dairy and poultry development; (iii) capacity building on implementation and monitoring and evaluation. However, the stated objectives clearly necessitated a longer term institutional development approach. A 2008 Quality Assessment of Lending Portfolio found that the Project, as designed, was unlikely to achieve its original physical targets and PDO by the Grant closing date, due to the PDO's, scope and targets being too ambitious given the institutional capacity, the country’s security environment, and the Project's three-year, emergency status.


Revised Design
Substantial
The Revised Project addressed the major issues of the results framework by revising the PDO to a more focused and measurable one with better links to outcome indicators and more realistic levels set for outcome targets. The revised project focused on activities in 11 Government-selected provinces and districts to enhance impact by mobilizing and focusing more intensively on participation and producer group formation, extension activities for adoption, value chain development, input supply and marketing. The following activities were cancelled with the restructuring: previously there were activities such as setting up a high level board on horticulture, this was dropped; dairy sub-sector specific activities were dropped; the project developed livestock extension packages that were to be disseminated in areas where intensive farmer mobilization efforts would be made; there were producer group formation efforts pre-restructuring but after the changes the focus came from bottom up rather than top down.


4. Achievement of Objectives (Efficacy) :

ORIGINAL PROJECT:
The original objective was : "stimulating marketable output of perennial horticulture and livestock by improving the incentives framework for private investments and strengthening institutional capacity in the agricultural sector ." Rated Modest.

Outputs:
  • 1,151 ha of new orchards (about 23% of the original end-2009 target) benefited 5,727 households, planting 445,350 superior varieties of seedlings using advanced techniques on selected farms.
  • 44% of target farmers adopted at least five best suitable practices (including winter/summer pruning, trellising and thinning), boosted by training and other project-financed inputs.
  • Seedlings were produced by contracted private nurseries and had an average 92% survival rate in 42 sampled orchards after 9 months (target 60% survival).
  • 4,000 ha of existing orchards (35% of target) were rehabilitated via technical advisory services to farmers on improved practices including IPM/pest control, on their land.
  • Value chain development (grapes) was piloted in Mir Bacha Kot District of Kabul Province in 2009 but market prices declined slightly over baseline (vs. 20% target increase in producer price). Planned expansion into stone fruits fell through due to security and bureaucratic issues.
  • Planned germ-plasm banks were not created because a competing, large-scale donor program was found to be already operational.
  • 7,000 women (28% of original target of 25,000) starting in 2009, received semi-intensive poultry package.

Intermediate Outcomes:

The results framework had insufficient indicators to measure the intermediate outcomes of improving incentives framework for private investments and strengthening institutional capacity. The ICR reported several activities as private investment incentives such as:
  • 114 government-supported Veterinary Field Units were privatized (95% of target), and
  • provided training, and other operational support to establish pay-for-service animal health services.
  • Liquid Nitrogen Plant was built to support Artificial Insemination activities, and arrangements negotiated for its operation/ownership, product distribution and pricing.
  • Extension services were established and expanded using the Farmer Field School mechanism for targeted horticulture producers. The Project recruited, trained and assigned 120 horticulture staff to target districts, and mobilized/organized Producer Groups.
  • Beneficiary cost-sharing was introduced: 25% for grape vine installation, 35% for grape, and 25% for almond, apricot, apple, peach and plum orchard establishment.
In terms of strengthening institutional capacity the ICR reported several activities/achievements, the most important were:
  • Ministry of Agriculture Irrigation and Livestock headquarters buildings were rehabilitated and staff trained.
  • Farmer Field Schools were established as the primary vehicle for extension services delivery.
  • Male/female Producer Groups were mobilized/organized and coverage expanded. The Project tried to reform- among the 1300 largely top-down, existing coops – those with potential to become sustainable Producer Groups, mobilizing 200 men’s and 182 women’s Producer Groups.


Outcome:
  • Tree productivity increased an average 110% for four major, marketable, perennial tree/vine crops (almond, apricot, grape and pomegranate) vs. the targeted 20% from over 4,000 ha of rehabilitated orchards (about one-third of the original end-2009 target).
  • Farm prices declined except for pomegranate.
  • Sales of fresh fruits from the project rehabilitated orchards increased an average 14.11% by end-2009 compared to an average increase among control households in the survey of 8.75%.
  • 71% of farmers achieved/exceeded the targeted 10% increase in production of four target crops. Net of control group effects, the average production increase was 9.75%.
  • About 3.7 million eggs were produced (31% of target for end-2009).
  • Meat production targets were deemed premature by the Mid Term Review and deferred to the restructuring period.
  • The Project planned to establish a model dairy processing unit involving 19,500 farmers and an incremental 40,000 liters/day by end-2009 but due to late field activation, and the realization that dairy results needed an extended lead time, only the Inception Study was completed and all other dairy activities were dropped.

The objective of stimulating marketable output of perennial horticulture and livestock is rated modest as some targets on increased production of perennial horticulture and livestock could not be met and activities on meat were postponed and dairy was dropped.


REVISED PROJECT:
The revised Project Development Objective was: " "to assist producer households adopting improved practices so as to increase horticulture and livestock productivity and production in Focus Areas". This is rated substantial.

Intermediate Outcomes:
  • 71% of farmers adopted at least two best horticulture practices, and adoption rates averaged 55% (vs. target 50%) over 16 separate practices covering improved production, new orchard management, and crop protection.
  • Recruited, trained and assigned to project sites 152 field extension workers (cumulatively under original and revised project) to advise farmers.
  • 92% of surveyed orchard farmers confirmed that HLP had assigned extension workers to their village and 90% reported frequent visits (min. weekly/monthly).
  • 90% of 919 sampled producers by end-2011 reported being trained by HLP livestock extension workers.
  • 560 poultry producers or 62% of a random sample, had adopted and fully-applied all seven improved practices, exceeding the 60% target.
  • The project provided training and set-up support of the 214 privatized Veterinary Field Units (129% of target).
  • A Farmer Field School system was fully functioning as the primary extension delivery mechanism by end-2011 and the beneficiary survey showed 93% satisfied with the project extension training in horticulture, livestock and farmer organization development.

Outcomes:
  • Productivity of rehabilitated orchards improved compared to control groups by end-2012: area and tree productivity of the four target crops rose an average 230% and 142% respectively over baseline exceeding targets at 70% and 48% respectively. According to the ICR this is attributed primarily to further systematization of extension delivery using the Farmer Field School approach, and the organization/mobilization of Producer Groups (p. 17);
  • Average production of target farmers for four key crops rose 41% vs. 10% target for end- 2012.
  • Cumulatively under the original and revised project, 4,952 ha of new orchards were established in 76 districts- benefiting an aggregate 23,923 target households.
  • Sapling survival rates averaged 76% vs. targeted 60%. 6 field inter-crops were established (such as garlic, alfalfa, tomato, water melon, Egyptian Clover and onion) helping to reduce/eliminate inter-cropped wheat, barley and maize, ecologically degenerative crops which compete with newly-planted seedlings (150 % of target).
  • Established another 17,800 semi-intensive household poultry production units distributing improved egg laying pullets to a total 25,000 women overall by end-2012, 100% of the overall project target over five years.
  • Backyard units produced 12.0 million eggs in 2010 (80% target) and 13.2 million in 2012 (88%). Hen productivity increased 67.4%/hen/year by end-2011 (target 15%) but began to decline due to women’s inability to maintain/invest in flock renewal/care as instructed by the project.
  • However, backyard poultry mortality rates averaged a 24 point increase by end-2012 despite widespread adoption of improved practices, attributed to initial delivery issues not farmer mishandling. The PDO indicator for that was 10 % reduction and was not achieved.
  • Meat production targets were dropped by the Additional Financing after repeated postponement (The target was 450 tons of meat production per year).
  • Better extension/health services for livestock through development of Veterinary Field Units, and improving animal care led to mortality rates decline an average (cumulatively for original project and revised project) 3.5 points for large and 8.5 points for small ruminants over baseline (vs. target 5 points and 10 points respectively).

Based on survey evidence, the Project overall made important steps in achieving the adoption of improved practices by mobilizing extension agents using the Farmer Field Schools and organizing producer groups, The horticulture-related outcome targets were in general achieved, and livestock outcome targets were partly achieved (there were still issues with the poultry production and, as noted, meat production targets had to be dropped). Due to achievements in terms of adoption of technologies and resulting productivity and production increases, particularly for the horticulture sub-sector, the overall efficacy rating for the revised objective is substantial.

5. Efficiency:

Rated modest on balance. The ICR provided economic analysis calculations for the original and revised project. The numbers are positive in both cases and treated cumulatively so no distinction in efficiency rating in terms of the original and revised project was made in this review.

According to the Memorandum and Recommendation Document (p. 9), it was estimated at appraisal that incremental income increase for participating farmers would be between 6 % and 140 % based on type of investment. Most tradable input and product economic prices were assumed to be equivalent to their market prices (except for labor where a shadow price of 0.7 was applied - to reflect the high underemployment in rural Afghanistan). All prices were assumed to remain constant in real terms throughout the life of the project. On this basis, the economic rate of return on total project investments over 20 years was estimated at 40.2% and, assuming a 12% discount rate for the expected costs and benefits, the net present value would be US$ 104.7 million.

The ex post ICR calculation on economic analysis mainly used similar assumptions on prices and costs to the ex ante analysis but incremental benefits were reportedly based on actual outcomes from data collected in several random samples of beneficiaries and control groups, including the baseline and the M&E annual output monitoring surveys. The ICR reported that (p. 53): "…the number of small ruminants by category (including animals sold) per year and at the end of the project cycle compared with the without project scenario, were derived introducing the project improved parameters, over and above those shown by the control population according to the data measured by the project M&E team". As mentioned in Section 10, the project's M&E system tried to compare the baseline to the current situation with and without the control groups, and from the information provided by the ICR, the control group comparisons seem comparable and therefore valid. Farm model budgets were also based on data provided by project horticultural and livestock teams. The ICR reported that (p. 19) incremental revenue increases were from the establishment of new and rehabilitation of existing orchards; and/or increased productivity of livestock and poultry. Ten farm models were used in the analysis combining crops and livestock activities and representing typical benefited households of different targeted project districts. Accordingly, the enterprise farm level aggregations showed that net income would have been expected to have increased between 7% and 41% above the without-project situation. However, it should be noted that this net income increase was based on farm budgets that include numerous assumptions. Also, given the questions on the sustainability of livestock activities (see section 7), there is a risk that benefit streams, assumed in the models to be sustained for 15 years, may not be sustained to the extent assumed. Furthermore, as stated in the ICR (p. 57), 72 % of those surveyed stated that the increase in livestock incomes were due to the improving economy, which raises concerns about attribution to the project. It seems likely that horticultural production would also have been significantly influenced by improvements in the economy.


The ICR efficiency analysis considered the results of the original project including implementation years 2007 – 2009, and separately, for the revised project - including the additional financing, as well as coming to a consolidated result for the overall project. Based on these assumptions the overall Economic Rate of Return was estimated at 21.6 % and the Net Present Value at US$ 40.8 million. Separate analysis of the original project period (2007 – 2009) provided an ERR of 12.9 % and a NPV of US$ 1.9 million, while the restructured project including the additional financing resulted in an ERR of 33.4 % and a NPV of US$ 38.9 million. Both the original and revised project economic analysis calculations showed satisfactory rates of return although the ERR for the original project was only slightly higher than the opportunity cost of capital.

However, there were operational and administrative inefficiencies. According to the ICR (p. 9) the Project incurred high overhead costs and low coordination between components and cross cutting activities (e.g., disconnect between Producer Group formation and investment activity) which eroded the amount of investment funds reaching farmers. Greater effort was needed to involve national professionals and phase out contractors as they had unreasonable/excessive cost. It was noted by the Bank team in mid-2011 that some 60-70 cents of every US Dollar were being spent on administrative costs with about 30% for expenditures in direct investments benefiting farmers. A National Horticulture and Livestock project indicator required that at least 70% of expenditures directly or indirectly reach/benefit farmers. Later there was a greater effort to address costs, particularly under the Additional Financing. The ICR did not provide information on the extent to which costs were reduced nor did it provide comparison data from other donors on overhead costs so the extent to which this situation is specific for this project or a systemic issue due to security reasons is unclear. The ICR cites (p. 9) two specific cases of high cost consultancies that were also not very effective.

On balance, given some concerns about the assumptions in the models, particularly sustainability (partly associated with the lack of credit - see discussion below under Risk), and the shares of attribution (project versus economy), and given questions about administrative efficiency and therefore cost effectiveness, efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
40.2%
100%
ICR estimate:
Yes
21.6%
100%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Original Project Outcome:
Moderately Unsatisfactory. Relevance of objectives was rated substantial due to relevance to country priorities and strategies, however relevance of design was negligible due to major shortcomings in the original results framework and the disconnect between objectives and design. Efficacy was rated modest due to project outcomes not meeting the ambitious targets and not being expected to. Efficiency was rated modest mainly due to operational inefficiencies and questions of attribution.


Revised Project Outcome:
Moderately Satisfactory. Relevance of objectives was rated high due to relevance to country priorities and strategies, relevance of revised design was substantial since, with the revision, the PDO was narrowed and more measurable outcome indicators and more realistic outcome targets were developed. Efficacy was rated substantial due to project outcomes achieved for horticulture sub-sector and achievements on the adoption of technologies. Efficiency was rated modest due to operational inefficiencies (high overhead costs) that continued during the extended period and also questions of attribution..

Overall Project Outcome:
The combined outcome rating of a restructured project is weighted according to the proportion of the Credit that was disbursed before and after the restructuring. As the original project had disbursed US$ 19.13 million, or 28 %, the combined outcome rating is moderately satisfactory. Overall, the project exhibited moderate shortcomings.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

This Review largely agrees with ICR's (p. 25) description of risks, however comes to a higher risk rating.


The major risk for sustainability is the political risk. As the ICR pointed out, the political and security situation in Afghanistan may erode particularly with the planned departure of NATO troops and this could lead to civil unrest and institutional failure. Even though government commitment is high, budget availability to sustain programs cannot be guaranteed.

Another risk identified by the ICR is the technical sustainability: although the project has shown that extension services and organization of farmers can help improve the horticulture sector, lack of access to credit by farmers is an important issue that jeopardizes sustaining and furthering these activities. Although the risk of lack of credit was known from the beginning, it was not mitigated and micro-finance institutions did not engage with the project. Another issue is that the sustainability of the project's semi-intensive “backyard” poultry model is difficult. Nevertheless, the National Horticulture and Livestock Project will try to achieve sustainability using better targeting, ensuring the capacity of village-level group leaders, better quality of birds, feed and TA / training, market facilitation, appropriate poultry health support, and greater scale. In terms of animal health services, the privatized Veterinary Field Units are still struggling. These privatized clinics are essential to the security of the livestock sector given the lack of a properly-functioning, government-supported animal health system and most of the privatized vets received training in strategic disease surveillance, disease reporting and vaccinations and about one-half so far - contracted by Ministry of Agriculture have assumed their field role under the new Sanitary Mandate System. However, the variable capacity to establish viable private animal health services for livestock breeders and to continue this activity independent of Government, raises concerns.

Environmental sustainability does not constitute a major risk beyond the overall security risk as it was noted by the ICR that it is an important focus of the National Horticulture and Livestock Project (which is a follow-on to this particular project) and it will continue to be included in technical messages to farmers and in government policies. The National Horticulture and Livestock Project will ensure that Ministry of Agriculture implements all aspects of its Pest Management Plan and that it mainstreams environmental concerns.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
As mentioned earlier, the Project was planned as a 3 year operation with a possible 3 year additional financing, however this approach was problematic. The fact that it was designed as an emergency operation without extensive background analysis, created design problems. Issues such as complex activities that were inconsistent with the level of institutional/professional capacity, the uniquely difficult Afghan environment and the time available. Assessment of the demand side and coordination with other donor funded projects in the country were not carefully considered. A clear identification of, and consultation with, project stake-holders or mapping of skills gaps/needs was not carried out. As mentioned in section 3.b, the design had a major shortcoming in terms of development objectives not being well linked to the indicators/targets. According to the ICR (p. 5): " The Quality Assessment of Lending Portfolio -1 based on interviews with the Bank team in 2008, concluded that while addressing relevant issues, design was complicated by an ambivalent concept of a larger 5-7 year project of which the three-year HLP would be the first stage, followed by an Additional Financing (also of three years). Further, the choice of a three-year emergency operation was guided by the intervention not being part of the Country Assistance Strategy but inserted later following an urgent request from Government. This was the first Bank-supported agriculture operation in Afghanistan in 28 years. Management’s agreement to this design was an error". The Quality Assessment of Lending Portfolio panel rated project design and implementation as Unsatisfactory.

The institutional framework called for the Ministry of Agriculture, Irrigation and Livestock to be the project
executing agency. An Implementation Management Support Team comprising specialist Facilitating Partners headed by a Lead Facilitating Partner would handle daily project implementation. Use of a Facilitating Partner was seen as a fast way to develop professional capacity but needed an exit strategy to assign nationals into key roles. However, this structure led to hiring of many expensive foreign consultants, and insufficient participation of national professionals in planning and decision-making. Design also did not specify a communications strategy/plan, and activities to disseminate information about project. An appropriate activity plan was not prepared resulting in the project attempting to move forward on multiple fronts simultaneously.

The ICR reported that (p. 6): "Project design only loosely reflected the lessons cited in the project document: (i) public regulatory, support and technical advisory capacity is needed to develop private horticulture, animal health, and production activities; (ii) coordinated technical assistance and training facilitate effective project start-up, promote sustainability and increase Afghan capacity both public and private; and, (iii) direct involvement of rural communities in the organization and management of production and marketing operations builds ownership and sustainability. Lessons specific to emergency agricultural/other operations in post-conflict environments such as post-Soviet countries, Kosovo and Central America might have provided additional guidance."

As mentioned in Section 7, the risks of lack of credit affecting the uptake of innovative practices and of in-cash contributions excluding poorer farmers were both realized but in practice, the mitigation measures were not effective and in key respects not even attempted. For example the design did not include mechanisms to facilitate engagement of micro-finance organizations to address lack of credit issue that eventually affected sustainability of certain activities. The risk assessment did not mention procurement capacity and its potential to de-rail the implementation timetable; as it turned out, procurement issues were a significant constraint.

Quality-at-Entry Rating: Unsatisfactory

b. Quality of supervision:
The evidence provided by the ICR (p. 26-27) on the quality of supervision suggests a moderately satisfactory rating. The ICR reported that there were well-staffed missions. The two stage Mid-term Review was effective and led to rapid restructuring to improve performance and a shift from an emergency to a sub-sector development approach. This built experience to support a scaled-up, follow-on project. Supervision of financial management, procurement and safeguards was reported to be strong. There was a focus on quality M&E, ensuring use of randomized sampling of target and control farmers. There was regular reporting anchored to the 2008 baseline.
However, there were a number of negative aspects including the following. The team was slow to acknowledge the project's bottlenecks and possible need for very early restructuring. The delayed Bank feedback on some draft Aide Memoires caused insecurity in decision-making for the implementation team. In addition to the ICR's points about the quality of supervision, this review finds the following: the Bank team could have followed up on the livestock activities more closely in order to identify and resolve bottlenecks especially with the project restructuring. The issues regarding the facilitating partner's handling of the livestock component was known at the time of restructuring and it appears that more could have been done. Another shortcoming was that the Bank team is as responsible as the Implementation Management Support Team for the development of national staff, essential to long-term sector sustainability. The team should have provided guidance or facilitated training on how to prepare national staff.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government gave strong support to the project throughout under exceptionally difficult country and sector circumstances. Although, the Government also could have intervened more forcefully to find solutions, together with the Bank, to the initial 18-month impasse. However, this Review agrees with the ICR assessment that the enormous stresses affecting Government’s broad operations and its general level of commitment to the project and its successor project warrant a satisfactory rating.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:
The evidence presented by the ICR (p. 28) regarding the Ministry of Agriculture, Irrigation and Livestock as the executing agency and Livestock and the Implementation Management Support Team's performances, yields, on balance, a moderately unsatisfactory rating. The ICR pointed out that the Ministry of Agriculture Irrigation and Livestock showed good performance in terms of leadership towards the project closing. There was good M&E performance under difficult circumstances with quality products/outputs despite the complexity of the Results Framework.

However, weak performance of the General Directorate of Animal Husbandry and Poultry (under the Ministry of Agriculture) which lacked commitment to or ownership of, the overall component, could not be improved as the Implementation Management Support Team failed to provide guidance on this problem.
Other issues mentioned by the ICR were: There was inadequate focus in addressing the retention and empowerment of, and delegation of greater responsibility to, national staff essential to long-term sector sustainability and to ensuring inclusive, participatory project planning, budgeting, and decision - making. Post-Restructuring, the Implementation Management Support Team should have strengthened information dissemination since the project's “brand identity” was weak; While building good relationships with other donors in the sector, the Implementation Management Support Team did not adequately mediate internal tension over roles and responsibilities; There was uneven oversight by the Lead Facilitating Partner of each Facilitating Partner and especially in some cases, their variable adherence to professional standards.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
PDO was not adequately aligned with the key performance and intermediate outcome indicators, i.e. the results framework did not include some outcome indicators important for the ambitious objective, for example assessing the incentives framework for private investments. These issues were mostly resolved with the restructuring. However, with the restructuring too many new indicators were included and the M&E system became very complex.

b. M&E Implementation:
M&E capacity was built within the Ministry of Agriculture Irrigation and Livestock's Policy and Planning Department, establishing a Management Information System incorporating fiduciary functions, and technical and economic monitoring for impact assessment. According to the ICR (p. 10) M&E was managed by the project M&E Specialist, Coordinator and team working with the Facilitating Partners and provincial monitoring teams. Field data collection was performed by contracted enumerators. The project M&E team also coordinated with other M&E efforts including the UK Department for International Development funded Capacity Building Project and the ASAP-funded National Agriculture Information System. The Bank approved a US$ 320,700 IDF grant in 2008 to enhance standardization of M&E and reporting practices for all Ministry of Agriculture projects including this project. The project produced the following: a Baseline Study (2008); Annual Monitoring Reports starting in 2009 covering the entire Results Framework and using baseline, random sampling and control group data;6 surveys and studies relevant to project objectives, activities and concerns, also using control groups and/or random sampling; and, good quality Borrower Completion Report and a Final Evaluation Study. This was a good achievement given the difficulties of field work.

a. M&E Utilization:
Dissemination was mainly through the Monitoring Reports and periodic workshops, as well as a popular radio program.. Project M&E outputs were valuable for the supervision effort, preparation of the follow up National Horticulture and Livestock Project, and the ICR. Monitoring data, and field surveys, were the basis for key project studies.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:
The Project had an environmental rating of B for safeguard compliance and triggered OP/BP 4.01 Environmental Assessment and OP 4.09 Pest Management. . Four specific issues were identified at appraisal for monitoring: Integrated Pest Management, Ethnic Minorities, Land Acquisition and Cultural Property. An Environmental and Social Safeguards Framework was prepared to guide implementation but was caught up in general delays affecting fieldwork initiation and monitoring. The project financed an Environmental Officer, Integrated Pest Management Coordinator, Pest Management Plan, a survey to determine pest and disease control practices, training on Integrated Pest Management application and pilot implementation in selected areas, and Integrated Pest Management training modules in the Dari language. Training modules were also prepared for Orchard Management and On-farm Water Use/Conservation. Land ownership issues affected the construction of some animal health clinics and location of the proposed dairy plant but were resolved through stakeholder consultation and agreement on a framework for handling land related issues. However the dairy plant, in any case, was not constructed. The project financed Liquid Nitrogen Plant was found ex-post to be constructed on a flood plain and was re-built on a more suitable site at Government expense. Cultural property performance was supervised by Bank missions but no issues were detected. The ICR did not provide any information on Safeguard compliance ratings.

b. Fiduciary Compliance:
The ICR provides information on fiduciary compliance (p 11). The Implementation Management Support Team's financial management team was able to install a Financial Management system for the project.. Performance ratings worsened over time, in parallel with the country's overall risk, and risk was rated Substantial. Internal controls of the project were problematic and an internal audit was not conducted until 2011 due to delayed recruitment of an internal auditor. The ratings in the unsatisfactory range were due to mostly the issue of duplicate claims, ineffective reconciliation processes, and outstanding operational and salary advances. Financial Management performance improved by the time of the Additional Financing. Audit performance was uneven with some reports delivered late and with qualified opinions, but audit issues were resolved in a satisfactory manner in collaboration with Bank FM/audit specialists. There were no outstanding audit issues at the time of ICR preparation.


In terms of procurement, capacity was a major issue during the first two years. This was mainly due to the project team’s lack of familiarity with Bank procurement procedures, continuing issues despite training as a result of the project’s complex technical and operational demands. Procurement ratings fluctuated and were repeatedly in the unsatisfactory range. Complex processes and intense Bank review of all documents slowed down project implementation. Contracting of a dairy Facilitating Partner was still under review by mid-2009, which was an important factor in canceling the dairy activities. Initially issues such as high turnover of project procurement staff, and replacement of the Lead Procurement Specialist, eventually were resolved and procurement improved with the preparation of a Procurement Guide and training. This Review agrees with ICR's statement that procurement activities should have been initiated well in advance of effectiveness.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Significant
High
The risk related to country security conditions is particularly high. Lack of credit is also a risk for sustaining benefit streams.  
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The Implementation Completion Report offers a number of lessons of which the following are the most important (with some reformulation of language)::

  • Project design as an emergency operation should be carefully weighed. Projects that require longer term sectoral improvement and institutional capacity building should be designed as regular operations with extensive background work. A clear identification of, and consultation with, project stakeholders, mapping of skills gaps/needs, and coordination with other donor funded projects in the country are all essential for such operations. The quick preparation needs of emergency operations limit this important preparatory work.
  • Heavy dependence on the Facilitating Partner model can be high-cost and a potentially low-return approach prone to problems; however if it is implemented well it can be valuable in countries where institutional capacity is limited. Facilitating partners and especially foreign expertise should be used to build capacity. TORs for such support should be clearly defined and agreed with the implementing agency, and advance procurement action taken so that TA teams are mobilized by project effectiveness. International Technical Assistance should be selective, temporary and for pre-identified technical issues, and with priority for local capacity building. The transfer of core technical and operational functions to trained nationals and sector agencies should be carried out at the earliest opportunity.
  • In post-conflict, “emergency” environments and for reconstruction efforts, establishing the foundation for transition from emergency recovery to medium to long term reconstruction and development is important. If productive activities with long lead/maturation times extending beyond project completion are essential, the expected phasing of activities and appropriate indicators for each phase should be established.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR was well written with detailed description of the challenges and issues and well articulated lessons. The economic analysis was thorough and the beneficiary survey evidence supported efficacy and efficiency assessments. However, some project costs and financing information was not clear. There was a discrepancy on total project cost and financing figures reported in the ICR. Also the reporting of the achievement of safeguard compliance was not complete.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions