Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Agricultural Services & Producer Organizations Project 2 (PSAOP2)

1. Project Data:   
ICR Review Date Posted:
Project Name:
Agricultural Services & Producer Organizations Project 2 (PSAOP2)
Project Costs(US $M)
 47.00  41.25
L/C Number:
Loan/Credit (US $M)
 20.00  20.76
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
 6.00  6.63
Board Approval Date
Closing Date
12/31/2010 06/30/2011
Agricultural extension and research (60%), Central government administration (20%), Agro-industry marketing and trade (20%)
Rural services and infrastructure (33% - P) Rural policies and institutions (33% - P) Participation and civic engagement (17% - S) Decentralization (17% - S)
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Ebru Karamete
George T. K. Pitman Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
The project was the second project of a three-phase 13-year Adaptable Program Loan, the Agricultural Services and Producer Organizations Program. The overall development objective of this Program (PAD, p. 4) was that producers, supported by agricultural services, utilize adapted innovations for increasing and stabilizing agricultural productivity, production and food security. The focus of the program was on: (i) institutional reforms to make agricultural services accountable, more demand-driven and cost-efficient, and to increase the participation in a number of functions hitherto executed only by the public sector; (ii) generating and adopting innovations so as to maintain and increase agricultural productivity and production in a sustainable way; and (iii) fostering empowerment, capacity-building, greater social accountability and inclusiveness of producer organizations.

The Project Appraisal Document (p. 5) states that the second phase project objective is:

    "to increase access for smallholder producers to effective and diversified agricultural services and innovations, with a view to diversifying and/or stabilizing their productions and increase their food security."

The Financing Agreement (p. 6 ) states the second phase project objective is:

    "to increase access of smallholder producers to sustainable and diversified agricultural services and innovations, in order to enhance agricultural productivity and to strengthen food security."

This Review uses the objectives stated in the Financing Agreement.

Not all triggers were met and the proposed third-phase APL did not proceed.

b. Were the project objectives/key associated outcome targets revised during implementation?

c. Components:
A. Support for the Agricultural Research System (Appraisal Estimate: US$ 10.0 million, Actual Costs: US$ 7.64 million). This component aimed to increase the capacity, as well as the relevance, efficiency, and sustainability, of the National Agricultural Research System (NARS) through:

      1. Strengthening the institutional guidance of NARS by increasing the capacity and redefining the role of its Steering Committee.
      2. Strengthening the capacity of the main research institutions, the Senegal Agricultural Research Institute (ISRA) and the Food Technology Institute (ITA) established under the phase-one project. The subcomponent was to support the rehabilitation of physical infrastructure (e.g. regional research centers); equipment, materials and vehicles for the research teams; technical assistance, studies, and training on scientific and methodological issues; information system, management of research results, publications and documentation; participation in national, regional; international research networks and workshops; and encouraging regional partnerships for capacity building in biotechnology.
      3. Supporting the evolution of the national Agricultural and Agroprocessing Research Fund (FNRAA) as the main national mechanism for both strategic applied agricultural research. This subcomponent aimed to provide funding through two windows. The first window was to finance strategic and applied agricultural research programs relevant to the priorities identified in the National Agricultural Research Framework. The second window was to finance beneficiary submitted demand-driven agricultural R&D subprojects which focus on resolving productive constraints on smallholders and on increasing their income.
      4. Strengthening linkages with other research systems through regional and international partnerships.
B. Agricultural Advisory Services (Appraisal Estimate: US$ 17.6 million; Actual Costs: US$ 17.91 million).
The first phase project supported the setting up of the National Rural and Agricultural Advisory Agency (ANCAR) and the agency's deployment of advisors in 142 rural communities. This second phase project aimed to support ANCAR to expand advisory services and programs to all 320 rural councils in Senegal. It had two main subcomponents:
      1. Supporting the strategic and methodological guidance of the national agricultural and rural advisory system through redeployment of ANCAR staff to establish a network of rural advisors. These advisors were to provide basic, non-specialized, advisory services; prepare advisory programs in all 320 rural councils; define methodological and technical support and manuals; and implement an integrated information system on agricultural and rural technologies.
      2. Strengthening the offer of agricultural advisory services. This aimed to support the emergence of a pluralistic network of specialized agricultural service advisors on high value-added activities for which the producers are willing and able to pay; and support the conception and implementation of rural advisory services that address the main technical constraints identified by producers (community seed production, livestock systems, production of quality groundnut, etc.). It also supported a study on the classification of family producers and their organizations in order to assess their capacity to generate demand, as well as their willingness and capacity to pay, for agricultural advisory services.

C. Support for Producer Organizations (Appraisal Estimate: US$ 8.3 million; Actual Costs: US$ 7.64 million).
This component sought to strengthen the capacity of producer organizations to represent and protect the interests of smallholder producers and to facilitate their access to technical and economic services in order to increase their production, food security, and incomes. It had three subcomponents:
      1. Strengthening social representation and inclusiveness of local and regional producer organizations. This aimed to support the extension of the Producer Organisations' Local Consultation Forum (CLCOPs) to all 320 rural council areas of Senegal. Newly established CLCOPs were to allow Producer Organizations (POs) to conceive and implement capacity-building subprojects and provide logistical and technical support to strengthen the role of rural development facilitators at the local level. The inclusion of women in decision-making at all levels was to be given special attention.
      2. Strengthening producer participation in policy formulation. This subcomponent will strengthen the capacity of POs to participate in the formulation, monitoring, and evaluation of sectoral policies. It aimed to support capacity building of PO leaders at the national, regional, and local levels; formation and communications programs for producers and their organizations; and creation of a demand-driven facility to strengthen the capacity and representation of POs’ national federations.
      3. Strengthening technical and economic capacity of smallholders and their organizations . This aimed to test solutions to the main technical and economic constraints faced by producers in the areas of production, processing, and marketing.
D: Support for Sectoral Coordination (Appraisal Estimate: US$ 10.5 million; Actual Costs: US$ 6.15 million).
This component sought to strengthen capacity in the Ministries of Agriculture and Livestock for policy formulation, planning, coordination, monitoring, and evaluation of the use of resources allocated to agricultural and livestock development. A particular aim was to support the definition and implementation of a sectoral medium-tern expenditure framework for both agriculture and livestock; the establishment of a subsectoral integrated information system; and the development of tools, procedures, and strategies for the rational use of the physical, financial, and human resources of the ministry and the implementation of the national agricultural and rural training strategy, at national and regional levels.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

    Project Cost :Total project cost at appraisal was estimated at US$ 47.00 million and actual costs were US$ 38.63 million, or 83% of the appraisal estimate. The underspending was because sectoral coordination only utilized 58% of its budget, agricultural research 76% and producer organizations 83%. The agricultural services component was only slightly over -budget (102%).

    Financing: IDA provided a credit US$ 20.0 million that gradually increased to US$ 21.67 million because of exchange rate variations. By closing US$ 20.76 million was disbursed and US$ 0.91 million was cancelled. IFAD provided cofinancing of US$ 6.0 million. Similarly this appreciated due to exchange rate variations and by closing US$ 6.63 million was disbursed.

    Borrower Contribution: At appraisal, the Borrower planned to contribute US$ 20.00 million; the actual contribution was US$ 13.86 million. While farmer beneficiaries were expected to contribute US$ 1.00 million, their actual contribution was zero.

    Dates: The closing date of the operation was extended for 6 months to end-June 2011. This was necessary to enable finalization of civil works to rehabilitate research stations of the Senegalese Institute of Agricultural Research and to consolidate achievements made under other project activities.

3. Relevance of Objectives & Design:

a. Relevance of Objectives:

  • The Program and the second-phase objectives are highly relevant to the needs of Senegal's agriculture sector. Senegal's Poverty Reduction Strategy Paper 2002-2005 highlighted the need to modernize and intensify agriculture, including by promoting the use of improved technologies. Agriculture remains a key sector of the Senegalese economy in 2014 and has the potential to activate inclusive growth, food security, employment and poverty reduction. The agriculture sector employs over 60% of the population, of which 60% are women. About 70% of the rural population depends on agriculture or activities related to agriculture for their livelihood. Despite the large share of employment, agriculture and livestock-raising contributes only modestly to GDP, with a rate varying of from 10% to 12% since 2010. The gap between the share of agriculture in the GDP and the share of agriculture in the labor force highlights the low performance of the sector. Even so, the sector has considerable potential. The climate is ideal for off-season horticulture and Senegal has good access to European markets. Domestic food markets are likely to expand with continued urbanization, and there is substantial scope for import substitution. There is potential also for large returns if yields can be increased with better access to land, improved fertilizer use, improved seeds, increased mechanization, and increase in the hectares irrigated to permit more planting periods
  • The project's objective were relevant to the World Bank's FY07-10 Country Assistance Strategy, particularly Pillar I: accelerated growth/wealth creation, the fourth of six outcomes from which was intended to be "promoting a modern and diversified agricultural sector.... with the ultimate goals of increasing agricultural productivity and diversification and improving household food security."
  • Project objectives remain relevant to the Country Partnership Strategy (CPS, FY2013-2017) that is anchored on one foundation and two focussed pillars. The Foundation, Strengthening the Governance Framework and Building Resilience includes, after improved public sector accountability, sustainable land and water management. And under Pillar I, Accelerating Growth and Generating Employment, measures are included to increased agricultural productivity and marketing through expansion of the horticultural subsector and expansion of the area under new technologies from 40,000 ha to 400,000 ha. The emphasis on food security makes the inclusion of the second-phase objective for food security relevant even though this is currently being addressed through the Global Food Crisis Response Program.

  • b. Relevance of Design:

  • The project's design is consistent with the objectives. The project logic summarized in the Results Framework is sound. The various activities under the four components were adequately linked to the Intermediate outcomes, which in turn were clearly important for achieving the project's objectives. Improving agricultural research was relevant as it is necessary to adapt technological innovations to the needs of producers. Strengthening agricultural advisory services in collaboration with public and private service providers was relevant as it is the main vehicle to link applied research and farmers. Strengthening producer organizations was relevant to boosting demand-led agricultural improvements that could inform and contribute to preparation and implementation of agricultural policies and programs as well as facilitating access to capacity-building and markets. Strengthening sectoral capacity and coordination among related organizations was relevant to ensuring sustainable management and budgeting. Inclusion of livestock and the population of nomadic herders, given the subsector's importance in Senegal, was also relevant.
  • Increasing agricultural productivity and diversifying agriculture was expected to contribute to food security by reducing risks due to drought, flood and pests. The project did not have any activities dealing specifically with the food security objective or in terms of targeting vulnerable groups such as pregnant or lactating mothers, infants or children as these are typically covered in social safety net programs.

  • 4. Achievement of Objectives (Efficacy) :

    "to increase access of smallholder producers to sustainable and diversified agricultural services and innovations, in order to enhance agricultural productivity and to strengthen food security."

    (i) Increasing access of smallholder producers to sustainable and diversified agricultural services and innovations: Modest


    • The National Rural and Agricultural Advisory Agency (ANCAR) lacked financial resources to scale up its field staff to provide national coverage; therefore only 205 rural communities out of 320 could be covered. The incremental coverage was only 63 communities compared to the 188 planned and thus only 35% of targeted increase in access was achieved.
    • The modest increase in access was a result of government failing to increase the agency's budget and Producers Organizations not being able to provide resources to cover operating costs. A result of this was that ANCAR had to rely on externally financed projects for funding and this meant that, rather than responding to farmers' demand for services, ANCAR was driven by the need to satisfy the demands of the external financing agencies, and generally there was no convergence of these two sources of demand. Consequently, the goals of ANCAR to oversee and regulate the national advisory system based on demand-driven services, as well as to support emergence of pluralistic network of service providers, was not achieved.
    • The number of producer organizations that signed contracts with advisory service providers other than ANCAR was negligible. The target was 20% and thus pluralism was not achieved.
    • Community seed production by participating producer organizations was less than targeted (actual 33 % of the local need vs. the target of 66% of the local need).
    • A total of 76 rural communities (target 320 over a baseline of 142) were served by at least one agricultural and one rural advisor. The incremental coverage was only 63 communities compared to the 188 planned and thus only 35% of target was achieved.
    • 40% of members of Producers Organizations signed contracts with agricultural advisors.
    • 85% of producers were satisfied with advisory services contracted (exceeding the target of 80%).
    • Among producer organizations, 40% have signed at least 1 contract with agricultural advisors (under the 50% target).
    • Among members of Producer Organizations participating in the first phase project, 72% have adopted at least one innovation (exceeding the target of 70%). However, the ICR did not provide description of the innovation for this outcome.
    • The project strengthened oversight of agricultural research mechanism and the capacity of related research institutions. Also the project strengthened Producers' Organizations in terms of contribute to preparation of agricultural policies and programs as well as establishment of and partnerships for cooperatives on groundnuts.
    • 320 local consolation forums for producer organizations were put in place.
    • A network of seed producers was established.
    • Ruminant vaccination increased from 20% before the project to 57% by the end of the project. Over the same time period vaccination against equine plague increased from 5% to 55%.
    • A network of decentralized financial services was established.
    • 5 producer federations (target 5) and rural regional consultation and cooperation frameworks received project support to improve efficiency and internal governance.
    • 37 new technologies (target 24) (including new tomato and sweet potato varieties, and residue of smoked fish as biologic fertilizer, etc.) were developed, tested, adapted and transferred to advisory services.
    • The National Agricultural and Agro-Processing Research Fund has been strengthened to establish links between research and extension institutions. Cofinancing of the National Agricultural and Agro-processing Research Fund reached 38% and exceeded the target of 30%. In addition, the Fund expanded its outreach beyond Senegal Agricultural Research Institute and the Food Technology Institute and disbursed 37% (target 15%) of its grants to agricultural faculties of universities.
    • Restructuring efforts of National Rural and Agricultural Advisory Agency included preparation and implementation of a deployment and a strategic plan.
    • The impact evaluation study found that 75% percent of the producers who received training were women, and women led 15% of Producer Organizations Local Consultative Fora and comprised 29% percent of Fora delegates. Fifty-eight percent of the producer organizations supported by ANCAR were women's organizations.
    • About 21% of farmers under 35 occupy leadership positions in the Producer Organizations Local Consultative Fora.

    (ii) Enhancing agricultural productivity: Modest.


    • The impact study found a 29% of increase in cropped area but only an 11% increase in production compared to villages not covered by the project. It does not indicate how much of the production increase was due to increased area, or increased crop yields. Total household agricultural production increased from an average of 1.60 tons in 2008 to 3.06 tons in 2010. However, the ICR does not indicate how much of this was due an increase in the area planted, how much was due to crop yield increase, or how much was due to changing the crop mix. Thus the stated production increase cannot be used to infer an increase in crop productivity.
    • The ICR (p.10) reported that productivity increased from 0.26 tons/ha in 2008 to 0.43 tons/ha in 2009; however, the time period was only 1 year, which is a very short time period to show productivity changes and it can be influenced by other factors such as weather. The ICR also reported that there were large fluctuations on productivity and production figures for different crops, and for some crops (e.g. corn and rainfed rice) yield decreases were observed.
    • Notwithstanding the above, the ICR (p.11) states that the yield reached for most crops (rice, sweet potato, maize, millet, onion) was half of the potential yield indicating that more progress could be made in terms of productivity growth.

    (iii) Strengthening food security; Substantial.


    • Calorie intake in poor households increased by 27%, and calorie intake per adult per day was about 3,400 calories in project villages compared to 2,700 calories for other villages.
    • According to the results of the impact study (ICR p.10 and p.16) the project contributed to reduction of 1 month of food deficit period for households (measured by the length of the hungry season which starts when last year's harvest has been consumed and ends when the new harvest begins, decreased from 4 months, 11 days to 3 months, 2 days). However, the ICR did not provide information on the control group results without the project areas.
    • 65% of producer organizations said that their food security had improved.

    Progress towards the third-phase APL.
    Only one of the four triggers required to launch the third-phase project was fully achieved, and one was partially achieved. Two were not achieved. In consequence, the proposed third-phase APL did not proceed.
    • Both the Agriculture and Livestock had Medium-Term Expenditure Frameworks since 2007 and used them for planning and resource allocation.
    • The requirement that at least 5 economic operations at the national level, and 10 at the regional level, have been successfully implemented by producer organizations, based on contractual arrangements was only partially achieved.
    • Advisory service outreach to all communities in Senegal was not achieved.
    • The requirement that the operational costs of basic rural advisory services and the National Agricultural and Agro-Processing Research Fund be provided by the National Rural Development Fund was not achieved. The Development Fund exists legally but does not yet have an administrative structure to manage funds.

    5. Efficiency:

  • A cost-benefit and economic analysis made at appraisal and project closing. The analysis focused on agricultural productivity, production and food security for with and without project interventions. The ICR (p. 33) cautions that reported financial and economic impacts cannot be attributed solely to the project. External factors including world food crisis affecting markets and prices, and incentives provided by the government's special programs for maize, rice and sesame played a role as well.
  • Farm household survey data as well as secondary data through meetings with project stakeholders was used for calculations. Project's benefits are as a result of adoption of technologies provided through research and advisory services under the project. For the crops sub-sector benefits included increase in income due to increase in cultivated area, yield and production. Six representative agro-ecological zones were selected for the assessment using a before and after analysis: production over the period 1995-2005 was compared with the production with the project 2006-2011. However, using different time frames for with or without project scenarios is problematic as it makes it impossible to control many variables including weather, macroeconomic factors such as prices, etc. that could impact production patterns, as well as yields during these different periods.
  • At the national level, based on these assumptions, the economic rate of return (ERR) was 38% and the net present value (NPV) was US$ 61 million. The financial rate of return for the same period was 40% with an NPV of US$ 68 million. These figures exceed the values projected at appraisal. The cost-benefit at appraisal was 1.30 and that at project closing was estimated to be 2.35. The relatively high rates of return on agricultural research and extension are fully in line with the Bank's experience elsewhere.
  • At the farm level the financial rate of return (IRR) was 25% and the net present value (NPV) was US$ 405 per farm. In comparison the appraisal values were 20% and US$ 319 per farm. The farm level cost-benefit at appraisal was 1.06 and that at project closing was estimated to be 1.14.
  • Administrative or operational efficiency was modest. The lack of budget support for ANCAR undermined achievement of project outcomes and most targets were missed. Despite spending all the project budget allocated, ANCAR only reached 63 communities compared to the 188 planned and thus only 35% of target was achieved. In addition, competition among some of the implementing agencies reduced the synergy of a programmatic approach.

    a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:

  • Rate Available?
    Point Value
    ICR estimate:

    * Refers to percent of total project cost for which ERR/FRR was calculated

    6. Outcome:

    Relevance of objectives is rated high, and that of design substantial. There were shortcomings in terms of increasing farmers’ access to sustainable and diversified agricultural services and innovations, that objective is rated 'modest'. Productivity was only modestly achieved, that objective is rated modest. Food security was improved and that objective is rated substantial. Efficiency is rated modest.

    a. Outcome Rating: Moderately Unsatisfactory

    7. Rationale for Risk to Development Outcome Rating:

    • Three outcomes have modest risks. The competitive research grant mechanism has demonstrated its usefulness. The empowerment of farmer organizations under the project will likely have a long-lasting impact with the social capital already growing. The government is also likely to continue the decentralization of its agricultural services provision with greater inclusion of the private sector.
    • The lack of sustainable financing for agricultural extension services poses high risks as does the lack of agreement in government on clearly define the mandate of the national Rural and Agricultural Advisory Service. While the Agricultural Framework Law foresees the establishment of National Rural development Fund it does not define the sources of funding.
    • There is a high risk that the small-holders will not be able to access the agricultural and rural advice that is key for improving their livelihood.

      a. Risk to Development Outcome Rating: Significant

    8. Assessment of Bank Performance:

    a. Quality at entry:

    Project design was based upon findings of a recent economic sector work and lessons from the first phase APL. Project implementation set up followed the same set up of first phase and covered 7 implementing agencies. There were several shortcomings in appraisal. Political risks were overlooked. Specifically the adversarial relationship between ANCAR and Senegalese Association for Promoting Grassroots Development was not detected. Consequently, the institutional setup for their collaboration was not designed thoroughly and clear mandates were not agreed. In addition the National Rural and Agricultural Advisory Agency lacked a financial plan to implement national coverage of its services, and it was not clear how the rising costs would be shared among government and the users of services. Also its governance was not adequately analyzed. Although it was planned to decrease government's capital commitment share, government continued to be the sole financier of the agency's operating expenses and decisions were dominantly taken by the government representatives. The M&E arrangements followed the first phase project, but there were some shortcomings with the M&E framework in terms of lack of adequately designed outcome and intermediate outcome indicators.

    Quality-at-Entry Rating: Moderately Unsatisfactory

    b. Quality of supervision:

      Some of the shortcomings of appraisal were addressed during supervision. These included completing the restructuring of extension agency, ANCAR, and ensuring the preparation and adoption of a strategic plan and program contract. In addition supervision convinced the government to prepare a strategic framework for agriculture. However, supervision lacked realism and seemed insensitive to the political realities. While the team constantly tried to establish uninterrupted dialogue with the major stakeholders, government frustrated these efforts through frequent turnover of ministers of agriculture and high turnover of directors general in ANCAR. While the Bank's continuous diplomatic dialogue helped to keep the project under implementation, the Bank seemed unaware of the government's growing lack of interest. As a result, even though ANCAR lacked sufficient staff and budget to cover the entire country because of government's lack of willingness and/or resources to address funding issues, the project was not restructured to reflect this reality. A consequence was that the triggers for the third-phase APL were not met.

    Quality of Supervision Rating: Moderately Unsatisfactory

    Overall Bank Performance Rating: Moderately Unsatisfactory

    9. Assessment of Borrower Performance:

    a. Government Performance:
    Government ownership languished during implementation. Specifically, although it was apparent that funding was necessary for national coverage of agricultural advisory services this was not approved. Also, due to frequent turnover of ministers dealing with agriculture, project implementation suffered from institutional instability. New ministers lacked time to familiarize themselves with the project, and the ministerial staff was involved very little in overseeing the project. More importantly, the ICR stated that (p.20) : "...The Ministry of Agriculture evidently did not consider the project as an instrument of national agricultural policy, but it did not officially request changes to the project, choosing either to ignore it (through the Special Presidential Programs for Agriculture) or counteract it (by encouraging the creation of producer organizations and an agricultural federation that competed with those supported under the project)." Furthermore, the Government provided only 69% of counterpart funds that was planned at appraisal. This was balanced by the Government’s contribution to the achievement of agricultural research activities; agricultural research has become more demand-driven and research institutions have become more relevant and efficient.

    Government Performance Rating: Moderately Unsatisfactory

    b. Implementing Agency Performance:

    There were seven main implementing agencies and their performances varied. Generally it was poor in the apex agencies but improved as the agencies mandates became narrower and more specialized.

    • Intersectoral Coordination, implemented by the ministries in charge of agriculture and livestock, was moderately unsatisfactory. In terms of coordination, the Ministry of Agriculture did not manage well the issue of lack of coordination and collaboration between the two main implementing agencies. Furthermore, decentralization was not completed as the Regional Directorates for Rural Development still lack the human and financial resources to carry out their regional planning functions effectively. The Ministry of Agriculture did not adequately supervise the project, provide implementation guidance to the executing agencies, and engage with the Bank supervision missions in an effective manner.
    • The performance of the National Rural and Agricultural Advisory Agency was marred by its lack of government support and funding. Thus national coverage for the planned 'one extension advisor per Rural Community' never materialized due to budget constraints. In addition, the Agency did not pro-actively try to encourage the emergence of private extension providers as planned by the project, and its funding problems caused it to abandon the demand-led approach to providing agricultural advice, and instead provide its services only through internationally financed donor projects.
    • Agricultural Research was implemented by the National Agricultural and Agro-processing Research Fund, the Senegalese Agricultural Research Institute, and the Food Technology Institute. These staff of these agencies performed satisfactorily due to their professionalism. However, both Institutes lost staff because their salaries became unattractive as a result of government policies. The ICR did not report whether this situation affected the implementation performance by these institutions.
    • Support for Producer Organizations was implemented by the Senegalese Association for Promoting Grassroots Development and its performance was generally satisfactory. It was pro-active in setting fora for producer organizations at the regional and local levels as well as strengthening Producer Organizations' capacity to influence agricultural policy formulation. Due to these efforts the rural population is better organized. However, its competition with the National Rural and Agricultural Advisory Agency undermined project effectiveness.

    Implementing Agency Performance Rating: Moderately Satisfactory

    Overall Borrower Performance Rating: Moderately Unsatisfactory

    10. M&E Design, Implementation, & Utilization:

    a. M&E Design:

  • The monitoring and evaluation system was based on the system established under the first phase project. Accordingly, each executing agency collected data to track progress against the verifiable indicators included in the Results Framework. The Technical and Fiduciary Coordination Unit consolidated the data. A baseline study of randomly selected family farmers as well as two impact evaluations were carried out.
  • One issue with the result framework indicators was that there was no outcome indicator to monitor the productivity increase objective. Also, the Project Outcome Indicator related to the food security was not clearly defined and there did not appear to be any Intermediate Outcomes directly linked to this.

  • b. M&E Implementation:

  • Baseline study got delayed and took place only in 2009, shortly before the Mid-term Review. The survey involved a comprehensive questionnaire administered to randomly selected 1, 580 farming families in 370 villages. The surveys had control group to distinguish between the beneficiaries and the non-beneficiaries. The baseline study and the two subsequent evaluations with the same respondents provided data on the project's impact with respect to increases in agricultural production, productivity, and incomes; gender issues; and smallholders' food security.

  • a. M&E Utilization:
    No information is provided by the ICR on M&E utilization.

    M&E Quality Rating: Modest

    11. Other Issues:

    a. Safeguards:
    The project was classified as Category B under OP 4.01 Environmental Assessment and only required a partial assessment. The project triggered OP 4.12 Involuntary Resettlement, OP 4.09 Pest Management, and OP 7.50 Projects on International Waterways. For OP. 7.50, the riparian states were notified on January 17, 2006, and no objection was raised. In 2007, the Directorate of the Environment updated its Environmental and Social Management Framework to ensure consistency with Bank policies. Also, two focal points were appointed to provide follow-up on environmental safeguards and national capacity in assisting producer organizations, and producers to follow environmental safeguards was strengthened. Extension agents were trained in integrated pest management, soil fertility, water management, soil erosion, and over-grazing.

  • The Producer Organizations' Local Consultation Forum network was used for awareness-raising campaigns on social and environmental issues and to reach a large number of producers. The training given by the Association for Promoting Grassroots Development covered issues related to collective action, social networks, conflict management at the local level, and social accountability. The project facilitated training in social and environmental issues for Producer Organizations. While the project triggered OP 4.12 Involuntary Resettlement, OP 4.09 Pest Management, the ICR did not provide information on implementation of the Involuntary Resettlement and Pest Management policies, nor did it provide any safeguard compliance ratings.

  • b. Fiduciary Compliance:
    Financial management was carried out by a team using appropriate management tools. Each implementing agency had its own financial and accounting system in place certified by an independent auditor. The Technical and Fiduciary Coordination Unit managed the designated account. Information required for accountability was issued regularly, and the conditions for disbursement were met consistently. Some persistent weaknesses in financial management remained to be addressed through more accurate budget monitoring, requiring a formal no objection from the Bank prior to financing any unexpected expenditure. The ICR rated financial management satisfactory, but more information on independent audit results would have been helpful.

  • The Technical and Fiduciary Coordination Unit followed the Bank's procurement guidelines and procedures, and consolidated, supervised, and monitored the procurement plans prepared by the different components. The implementing agencies set up their own procurement systems under the overall guidance and quality control of the procurement specialist of the Technical and Fiduciary Coordination Unit. Improvements on good procurement records management were still needed. The Technical and Fiduciary Coordination Unit procurement specialist left the project during implementation and the position could not be filled, and so the quality control was no longer effectively ensured.

  • c. Unintended Impacts (positive or negative):

    d. Other:

    12. Ratings:

    IEG Review
    Reason for Disagreement/Comments
    Moderately Satisfactory
    Moderately Unsatisfactory
    Relevance of objectives is rated high, and that of design substantial.Two of the three objectives are rated modest and the third objective is rated substantial. Efficiency is rated modest. 
    Risk to Development Outcome:
    Bank Performance:
    Moderately Satisfactory
    Moderately Unsatisfactory
    Quality of entry is rated moderately unsatisfactory as is supervision. 
    Borrower Performance:
    Moderately Satisfactory
    Moderately Unsatisfactory
    Government performance had significant shortcomings, and implementation agencies performed with moderate shortcomings. Information is lacking on safeguard and fiduciary compliance. According to the Harmonized ICR criteria, when one rating is in the satisfactory range and the other in the unsatisfactory range, the overall rating will be determined by the outcome rating. 
    Quality of ICR:
    - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
    - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

    13. Lessons:
    The ICR offers a number of lessons of which the following are the most important (with some reformulation of language):
  • Institutional reforms in the agricultural sector requires long-term and high level involvement by the Bank and other donors. Since such reforms are generally politically sensitive, frequent political dialogue is required at the highest level, over the course of an Adaptable Program Loan. Such efforts should be supported not only by the technical department but also at the level of the country director. Collaboration with other donors is also important.
  • Producer organizations are a crucial part of agricultural services provision and they should be supported and involved for successful reform of the sector. Producer organizations should be supported if they are to become partners with government in formulating agricultural policy and implementing agricultural programs at the national, regional, and local levels. The effectiveness of the agricultural extension and research system depends to a large extent on the strength of producer organizations.
  • Evaluating the impact of agricultural services projects is a complex and methodologically challenging task but can be overcome by hiring universities to carry out the work. In many countries, the capacity to do such studies is very limited or nonexistent. Partnerships between local specialized universities and external universities with strong evaluation capabilities might be the best solution. These partnerships should be active from the very beginning of the project to ensure that baseline studies are undertaken on time.
  • Project design should clearly spell out institutional implementation arrangements including amounts and sources for budgets. The project experience showed that lack of clear mandates for the implementing agencies may create competition between the agencies. Also the design lacked a financial plan to implement national coverage of extension agency's services, and it was not clear how the rising costs would be shared among government, and the users of services.

  • 14. Assessment Recommended?


    15. Comments on Quality of ICR:

    The ICR was comprehensive with a candid report of implementation challenges. It used an impact evaluation to examine outcomes in relation to the development objectives. However, quantitative evidence on outcomes from the impact evaluation could have been more rigorously analyzed in terms of control groups, etc. Economic analysis calculations also had some methodological issues, particularly regarding the validity of data used for calculations. The ICR also did not provide sufficient information on implementation of the Involuntary Resettlement and Pest Management policies and the extent of compliance. Also, while the ICR rated financial management satisfactory, more information on audit outcomes would have been helpful. The ICR is rated satisfactory, but only barely so.

    a. Quality of ICR Rating: Satisfactory

    © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions