Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Agricultural Transition


  
1. Project Data:   
ICR Review Date Posted:
06/27/2014   
Country:
Colombia
PROJ ID:
P082167
Appraisal
Actual
Project Name:
Agricultural Transition
Project Costs(US $M)
 32.00  54.39
L/C Number:
L7313
Loan/Credit (US $M)
 30.00  29.39
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
 0  0
Cofinanciers:
Local Sources of Borrowing country
Board Approval Date
  06/28/2005
 
 
Closing Date
03/31/2010 12/30/2011
Sector(s):
Agricultural extension and research (69%), Animal production (13%), Central government administration (12%), Crops (6%)
Theme(s):
Infrastructure services for private sector development (29% - P) Rural services and infrastructure (29% - P) Export development and competitiveness (28% - P) Rural policies and institutions (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Hassan Wally
Kristin Hallberg Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
According to the Project Appraisal Document (PAD, p. 7) the project development objective (PDO) was:


    "to strengthen the National Agricultural Science and Technology and Sanitary and Phytosanitary Systems by supporting the joint participation of both the public and private sectors, through the mechanism of production chains, thereby contributing to the competitiveness of Colombian agriculture and improving the accessibility of export-potential products to international markets."

According to the Loan Agreement (p. 20) the project development objective (PDO) was:
    "to strengthen the Borrower’s agricultural national science and technology system and SPS (Sanitary and Phytosanitary Measures System) by supporting joint participation of both public and private sectors through the mechanism of production chains and thereby increase the competitiveness of the Borrower’s agriculture sector and improve the accessibility of export-potential products to international markets."

IEG will evaluate the project against the PDO as stated in the Loan Agreement in accordance with the Guidelines for Reviewing World Bank Implementation Completion and Results Reports.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1. Knowledge Generation and Innovation (Appraisal estimate: US$22.1 million, actual cost: US$18.8 million). To support the provision of technology and innovation through strengthening the agricultural production chain actors including farmers, ago-entrepreneurs, and trader associations, co- financing and implementation of research and development R&D, as well as the preparation of participatory and demand driven R&D agendas around certain production chains and the implementation and co-financing of these agendas through a Competitive Fund. The component would finance: (i) the development of R&D agendas for twenty production chains, which would entail the carrying out of the prospective studies for these agendas as well as the evaluation and strengthening of the capacity of chains to develop, co-finance and implement these agendas; (ii) the operation of a Competitive Fund that would co-finance production chains’ R&D sub-projects which are consistent with the prospective agendas; (iii) capacity building within the Ministry of Agriculture and Rural Development (MARD) to manage the allocation of public resources to the agendas already defined and to the development and implementation of new production chains’ R&D agendas.

2. Strengthening the National Sanitary and Phytosanitary Measures System (Appraisal estimate: US$7.8 million, actual cost: US$7.9 million). To strengthen the Sanitary and Phytosanitary Measures System, enabling it to respond to the sanitary standards of international markets and thereby enhance the admissibility of agricultural products to these markets. The component would finance the following activities: (i) training professionals and technicians attached to the SPS systems; (ii) capacity strengthening (i.e. training, development of procedures, equipment, infrastructure) to mitigate the chains-specific sanitary barriers to beef products; (iii) capacity strengthening (i.e. training, development of procedures, equipment, infrastructure) to
mitigate the chains-specific sanitary barriers to fruits and vegetables; (iv) designing and implementing a national SPS information network; (v) designing and implementing an authorization system allowing for Audit and/or Test and Diagnosis Laboratory activities to be delegated to a third party; (vi) technology transfer on best practices to improve the sanitary and phytosanitary conditions of agricultural production. To improve the impact of the project, needs identified in the SPS system were prioritized vis-a-vis requirements of two production chains (beef, and uchuva Physalis pevuviana or Andean berry) showing a competitive potential particularly in the US market and which would have a spill-over effect to other chains. The beef chain was chosen within the livestock sector because its products are highly complex in sanitary terms, not only because they face zoo-sanitary barriers but also other food safety-related barriers. Although the investment plan may have been designed on the basis of overcoming barriers for a particular fruit (uchuva), its range covers the needs being faced by a large share of the fruit and vegetables chain to successfully access international markets.

3. Project Management, Monitoring and Evaluation (Appraisal estimate: US$2.1 million, actual cost: US$3.3 million). To ensure proper project management and execution this component would finance a Project Coordination Unit which would be attached to the MARD’s Technology Development and Sanitary Protection Directorate (DDTPS) and be responsible for the general management of the project. A Project Coordinator would be appointed and would be reporting to the DDTPS Director on matters related to project implementation. The PCU would be composed of a Project Coordinator, a Financial Specialist, a Procurement Specialist, an M&E Specialist, two Knowledge and Innovation Specialists (one responsible for coordinating Sub-component 1 activities and other responsible for coordinating Sub-component 2 activities of Component 1 of the Project), a SPS Specialist and a Secretary. In addition, MARD would assign at least two counterpart officers to work in close relationship with PCU’s M&E and Financial Specialists, as a means to strengthen MARD’s institutional fiduciary capacity.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost. According to the PAD (Annex 5) the total project cost at appraisal was estimated to be US$32.00 million and identifiable taxes and duties were US$2.00 million, so the total project cost, net of taxes, was US$30.00 million. The actual project cost was US$54.72 million. A reallocation of funds was undertaken in 2010 based on: (i) the allocation of the “contingency” expense category to specific project components and activities; (ii) that the objectives and results of activities related to the R&D agendas of Component 1 were achieved (and surpassed) with fewer than expected resources spent; (iii) the extension of the closing date required additional resources towards Component 3; and (iv) that the execution of Component 2 had higher than budgeted costs. The reallocation increased the amount of resources to Component 2 and 3 by US$600,000 and US$1.3 million respectively; and reduced Component 1 by US$150,000.

Financing. An IBRD Loan of US$30.00 million was approved and US$29.39 million was disbursed. At closing US$0.61 million was cancelled.

Borrower Contribution. At appraisal the borrower was not expected to contribute any counterpart funds, however, the ICR (Annex 1) reported that the borrower contributed US$15 million of counterpart funding to support sub-projects. In addition, it was estimated at appraisal that local sources of the Borrowing country would contribute US$2.00 million for cofinancing sub-projects, actual contribution was US$10.00 million (ICR, Annex 1).

Dates. The project was scheduled to close on 03/31/2010; however, the closing date was extended twice for a total period of 21 months to 12/30/2011. The extension was to allow for the completion of individual sub-projects under component 1 and for the completion of the National Biosafety Laboratory Level 3 Agriculture (NBL3Ag) under component 2 (ICR, p. 4).


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High. At the time of appraisal, the Colombian agricultural and agro-entrepreneurial sector accounted for 21 percent of aggregate GDP, 25 percent of export revenues, and 30 percent of job creation in the country, employing more than 4.5 million. However, Colombia was ranking lower in terms of sector competitiveness when compared with countries at similar level of development. Project objectives were substantially relevant as Colombia had embarked, along with its partner countries in the Andean Region, on the negotiation of a Free Trade Agreement with the United States. Improvements in productivity, competitiveness and phyto-sanitary regulatory aspects of export markets would likely improve the benefits of the FTA.

The objectives were aligned with the first pillar of the (2002-2006) Colombian National Development Plan which called for sustainable economic growth and employment generation by implementing labor and pension reform, promoting competitiveness, innovation and technology, fostering trade integration and increasing private participation in the delivery of public services and infrastructure (PAD, p. 1). Objectives were also in line with the Bank's Colombia Country Assistance Strategy (CAS-2003) whose first priority was for achieving rapid and sustainable growth. In addition, objectives would also contribute to the second priority which aimed to ensure that all Colombians benefit from growth, particularly those living in poverty.

At project completion the objectives remain relevant and in line with the Government of Colombia's plans to strengthen productivity, innovation, and entrepreneurship and its commitment to scale up investments in science, technology and innovation, as reflected in the (2011-2014) National Development Plan and the (2010) Plan Vive Digital. Objectives are also in line with the Bank's Colombia partnership strategy (2012-2016) which among other things aims to support sustainable growth with enhanced climate change resilience; and inclusive growth with enhanced productivity.

b. Relevance of Design:
Modest. Design included an ambitious statement of objectives that was not completely aligned with project activities. To increase competitiveness and improve the accessibility of export-potential products to international markets the design featured activities that focused on strengthening the National Sanitary and Phytosanitary Measures System (SPS) as well as investing in agriculture research and development (R&D) through a competitive fund that emphasized transparent public investments and leveraged private sector funding. This arrangement ensured continuity of sector investment priorities beyond political cycles. Design supported investing in agriculture R&D through a value chain approach that would allow prioritization of activities and involvement of stakeholders. Also, design flexibility would enable the project to become integrated within the Technological Development and Sanitary Protection Directorship (DDTPS) of the Ministry of Agriculture and Rural Development (MADR).

Overall, project components described intended activities and inputs, however, there was no prioritization of activities and investments and a lack of coordination among activities and between components. There is a gap between the project activities and its ambitious PDO. Moreover, there was a gap between the PDO indicators and the PDO itself. Design also suffered from the lack of a clear mechanism to establish inter-relationships among sub-projects funded through the competitive fund. In addition, design suffered from the absence of a systematic approach to transfer technologies to farmer groups. Furthermore, design seemed to have overlooked the importance of financial resources and accessible lines of credit for small farmers to adopt new technologies. Design should have focused more on applied research rather than basic research activities. Finally, design underestimated the complexity for achieving international admissibility of beef products as well as the institutional capacity required to handle competitive R&D funds and supervise R&D investments. This resulted in delays on SPS investments and the concentration of funds in two large institutions that captured most of the competitive funding (ICR, pp. 5&9).


4. Achievement of Objectives (Efficacy) :

The PDO includes three sub-objectives as follows:

(a) Strengthen the Borrower’s agricultural national science and technology system and SPS (Sanitary and Phytosanitary Measures System). Substantial.

Outputs:
  • 591 sub-projects were implemented: 117 Bank-supported sub-projects; and 474 MADR-supported sub-projects (160% more than the original target of 225). 32 Departments and 21 supply chains benefited from the project-supported sub-projects. Level of co-financing from participants in sub-projects reached the expected 25% of sub-project costs.
  • 7 MADR staff permanently working on agriculture R&D activities (10% more than originally planned).
  • 386 trained in animal and plant health and authorization systems (74% of the original PAD target of 521).
  • First phase of the design of the national SPS information network has been completed, but the implementation of the inter-connectivity of the information systems of each institution is still pending (60% accomplished).
  • Colombian Agricultural Institute (ICA) SPS control and surveillance system was approved and activities were authorized, confirming the authorization system’s operation.

Outcomes:
The project introduced competitive grant funding mechanism as a model for allocating scarce public sector resources for agriculture R&D, and for establishing new relationship between the different actors along the supply chains to establish R&D priorities and investment plans. The project also supported upgrading SPS control and surveillance and supported the establishment of an in-country National Biosafety Laboratory Level 3 for Agriculture (NBL3Ag) to address foot and mouth disease--safeguarding a livestock industry that represents 20% of agriculture GDP. However, there are concerns regarding the type of research supported by the project as the Mid Term Review pointed out that the project needed to foster more applied research rather than basic science research. The project also fell short on achieving some targets related to the strengthening and operation of laboratories and inspection of beef products as well as the presentation of protocols to the EU on the Mediterranean Fruit Fly and implementation of residue control plans. Subsequent communication with the project team revealed that the NBL3Ag laboratory was expected to be fully operational by the end of October 2012. The stakeholder survey and impact evaluation results (ICR, Annex 5) pointed out that about 50% of the total costs of sub-projects were provided through private sector funding. In addition, the same study (ICR, Annex 5) pointed out that research organizations, as a result of project support, successfully generated technologies that were transferred to and applied by farmers. Overall, it seems that the project positioned the Colombian agriculture for future gains in productivity with a better chance for accessing international markets.

(b) Increase the competitiveness of the Borrower’s agriculture sector. Modest.

Outputs:
  • 460 extension agents trained in Good Agriculture Practices (GAPs) and Good Manufacturing Practices (GMPs) (PAD target: 150)
  • 25 agriculture supply chain R&D agendas were developed (25% more than the original target).
  • Installed capacity within 10 agriculture supply chain organizations to prepare and prioritize agriculture R&D agendas.

Outcomes:
The project met a number of output targets, however, it is challenging to gauge the impact of the project on agricultural productivity and competitiveness given that no sub-project has yet completed the process of transfer and no massive adoption of new technologies has taken place (ICR, p. 34). The project also lacked relevant PDO level indicators to gauge the impact of strengthening the agricultural national science and technology system on competitiveness. In addition, less than 30% of farmers knew the agriculture R&D instruments available to them through the public system and of those who knew about such instruments, less than 50% used them (ICR, p. 19). The absence of a systematic approach to transfer technologies to farmer groups undermines the achievement of the PDO and cast doubt on the adoption of new technologies.

(c) Improve the accessibility of export-potential products to international markets. Substantial.

Outputs:
  • Required conditions for admissibility of beef products were met, including the establishment of 8 control posts, and recognition of Colombia as free of foot and mouth disease (FMD), where vaccination is practiced. The strengthening of the laboratories of ICA and INVIMA related to beef biosafety and other aspects were mostly completed, with the exception of the certification of the NBL3AG (expected by end of 2012).
  • Required conditions for admissibility of fruit and vegetables (particularly uchuva) were mostly met, with the exception of the presentation of protocols to the EU on the Mediterranean Fly which was not completed. .


Outcomes:
The project main outcome indicator pointed to an annual increase in agricultural exports by 13% during the life of the project, however, it remains unclear to what extent this increase could be attributed to the project activities. Beef production increased by 25% during the same period, but exports decreased due to a border problem with Venezuela. Uchuva exports were maintained at 4,300 metric ton over the project period. The ICR provided limited evidence on the impact of the project on improving the accessibility of export-potential products to foreign markets. That said, it is expected that upgrading the SPS and establishing the NBL3Ag could potentially enhance the admissibility of Colombian agricultural products in the future to international markets. Subsequent communication with the project team revealed that the Colombian government signed the free trade agreement with the United States. This is viewed positively since it opens a sizable market for Colombia's agricultural exports and provides an added incentive for producers to improve competitiveness.

5. Efficiency:

The PAD (Annex 9) provided an ex-ante economic analysis for investments in agriculture R&D focusing on seven production chains (corn, cocoa, rubber, sugarcane, plantain, meat and tropical export fruits). The analysis used the Dynamic Research Evaluation for Management (DREAM)*. Internal Rates of Return (IERR) ranged from 23% to 144% depending on the chain, type of investments and parameters used for analysis. A 20 year investment horizon was covered.

The ICR (Annex 3) pointed out that it was not possible to compare ex and post economic analysis because the value chains funded through agricultural R&D investments were not the same as the ones chosen at appraisal. The only exception was the meat and uchuva value chains which were analyzed ex-ante. The ex-post economic analysis covered 56% of project direct investments in agriculture research and development. The analysis did not include resources devoted to the research agendas (6,5%), to the strengthening of the National Sanitary and Phytosanitary Measures System (26%) and to the Project Management and Monitoring and Evaluation (11%). The ex-post economic analysis focused on 7 production chains (pisciculture, aromatic and medicinal plants, forestry, biofuel, fruits, guadua (bamboo) and beef) which represented more than 70% of R&D investments and 68% of sub-projects. The analyses used the DREAM program to analyze 12 sub-projects over 20 year period, including time required for generation, dissemination, adoption of technologies being evaluated, and assuming a linear form of adoption. The analysis showed that IERRs ranged from 14 to 99%. The success probability of the sub-projects ranged between 50 to 100% depending on the stage of research, type of sub-project and the scientific and corporate entity involved. The impact evaluation results (ICR, Annex 5) stated that the sub-projects showed a positive and significant economic impact in the variables related to production including: production per hectare, income per hectare and agriculture value added. However, no significant impacts were found in other variables such as access to markets or exports.

IEG finds a number of concerns that cast doubt on the financial efficiency analysis, first, the results of the analysis are not representative and cannot be extrapolated to the 117 sub-projects funded by the project, second, the analysis did not include the financial resources required for dissemination, extension and large scale production of sub-project activities which according to the ICR (p. 36) if included would decrease the value of IRRs; third, the adoption levels used in the calculation seem overly optimistic given the absence of an extension activity; fourth, there was no baseline available for the sub-projects approved and implemented by the project; fifth, the sample of chosen sub-projects was not a fully randomized sample which means that sample bias could not be ruled out.

In terms of administrative efficiency, project management costs significantly exceeded the appraisal estimate (by 57%) and project closing date was delayed by 21 months (a 37% time overrun) in addition to fiduciary concerns surrounding the construction of the NBL3Ag laboratory. The construction and completion of NBL3Ag suffered from implementation delays stemming from weakness in the design, work amounts and budget estimation for the construction, and lack of specific experience of the team responsible of the monitoring of the construction. Implementation of the project activities were also affected by the absence of a format for sub-project progress reports and the unavailability of the Project Management Information System (SIGP) for the first call for proposals which resulted in a 2 year delay in the delivery of the first prospective agendas. The Project Management Information System (SIGP) implementation for the public calls operations was only feasible in 2008. During 2006 and 2007 all operations were done manually, which produced a lack of efficiency and control in the procedures. Further, public calls of 2007 and 2008 were highly demanding in human resources and time, which triggered the allocation of non-planed resources to hire personnel, computers, and communication networks. Also, the training of the sanitary agencies’ officials was impossible to execute according to what was envisioned in the PAD, given high Government guidelines that did not authorize the ICA and INVIMA officials to go out of the country. Finally, it took 2 years for the design and implementation of the management information system to become fully operational which also required an important investment of significant credit resources.

Efficiency is rated modest.

* a menu-driven software for evaluating the impacts of agriculture R&D developed by the International Service For National Agriculture Research (ISNAR).

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
%
56%
ICR estimate:
%
56%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Outcome is rated moderately satisfactory. Relevance of objectives was rated high, but relevance of design was rated modest where the project suffered from design shortcomings such as the absence of a systematic approach to transfer technologies to farmer groups (see section 3b). Efficacy of both the first and third sub-objectives was rated substantial, but modest for the second sub-objective (see section 4). The project introduced competitive grant funding mechanism as a model for allocating scarce public sector resources for agriculture R&D, and for establishing new relationship between the different actors along the supply chains to establish R&D priorities and investment plans. The project also supported upgrading the SPS control and surveillance and the establishment of an in-country National Biosafety Laboratory Level 3 for Agriculture which would potentially enhance the admissibility of Colombian agricultural products to international markets. However, there is limited evidence on the impact of the project on competitiveness and improving the accessibility of export-potential products to foreign markets. Overall, the project seems to have positioned the Colombian agriculture for future gains in productivity-if a system for dissemination of technologies is developed, with a better chance for accessing international markets especially after signing the free trade agreement with the United States. Efficiency was rated modest. Project closing date was extended by 21 months and there were concerns with regards to the financial analysis and administrative efficiency (see section 5).

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The absence of a systemic approach to transfer technologies generated through sub-projects to farmers seriously undermines the project. Such shortcoming could impact financial contributions from different stakeholders who might develop a negative view of the project due to the absence of impact at the field/productivity level. The delay in getting the NBL3Ag laboratory certified and operational casts further doubt on sustainability. The ICR (p. 7) highlighted that financing the NBL3Ag laboratory was justified, but should have been viewed as a potential risk given the complex nature of such investment. The ICR (p. 20) pointed out that there was a clear commitment from the Colombian Agricultural Institute (ICA) to operate and maintain the NBLAg3 lab. It is not clear how the R&D research agenda would be funded in the post completion stage given that the Science and technology agenda is not only shaped by MADR, but other institutions as well (ICR, p. 7). In addition, evidence form the evaluation of the competitive fund revealed that some of the investments were driven mainly by the R&D institution rather than the producer organizations, which could have implications for beneficiaries’
willingness to invest in and adopt the technologies produced. There is also a concern regarding the future investments in R&D especially after integrating the PCU and PTA methodology for the management and supervision of subprojects into the institutional structure of MADR. Such investments would then be dependent upon the research institutions' own investment and budget plans as well as the level of participation of producers in the subprojects.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
The Bank correctly identified an operation that was in line with the borrower's strategy. The project sought to bring the Bank's experience in terms of public-private partnerships in agricultural R&D and guidance in establishing SPS standards. An innovative approach to reform public funding of agriculture R&D was introduced and correctly focused on key areas needed to improve the competitiveness of the agriculture sector of Colombia.
However, the project design suffered from a disconnection between its ambitious objectives and project activities (ICR, p. 20), for example, the project had no activities geared towards technology dissemination. The project also suffered from vagueness of project investments combined with problems with attribution of key project indicators (see section 10a). There was no prioritization of activities and investments and a lack of coordination among activities and between components (ICR, p. 20). While relevant lessons for successful agricultural development projects were identified at appraisal stage, their reflection in project design was uneven. Most notably the absence of a systematic mechanism to reach out to small farmers and absence of indicators that could gauge the impact of R&D activities on productivity at the farm level (ICR, p. 5). In addition, the construction of a highly complex technological infrastructure such as the NBL3Ag laboratory should have been fully envisioned at the appraisal stage to ensure better planning with risk identification and proper contingent measures.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
The Bank supervision team undertook 15 missions in 7 years. Missions were well staffed and efforts were made to add indicators to measure progress. Supervision was pragmatic and responded to changes in Government priorities. The team sought the direct engagement of the Country Management Unit with authorities to address problems with NBLAg3 and the investigation by the General Accounting Office (ICR, p. 21). To address design shortcomings, the team supported building the capacity of project coordination unit to develop a program-based M&E and safeguard system (ICR, p. 21).

IEG finds that the team should have opted for a Level 1 Restructuring of the project given the disconnect between the development objectives and project activities and the absence of relevant indicators combined with limited focus on dissemination and adoption of technologies.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government was supportive of the new demand driven approach for agricultural R&D and adopted the project's approach in allocating R&D funds. However, support for the same approach waned under the new Government. According to the ICR (p. 22) the NBL3Ag laboratory remains to enjoy strong budgetary support. However, the project suffered from effectiveness delays that contributed to the extension of project closing date.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
Changes in the political leadership of the Government prompted changes in the project coordinator. This transition in leadership led to relatively long periods of inactivity towards the end of the project (ICR, p. 22). The ICR (p. 22) highlighted that the implementing agency showed committed effort to launch the call for agriculture R&D proposals in spite of challenges in the earlier years of the project stemming from the absence of a format for sub-project progress reports and the unavailability of the Project Management Information System (SIGP). The ICR (p. 22) highlighted the PCU efforts to deliver training to technical teams responsible for agriculture R&D investments particularly on fiduciary aspects, safeguards and M&E. According to the ICR (p. 22) PCU tried to maintain efficiency and management oversight of the project execution in spite of capacity gaps especially with the construction of the NBL3Ag. The ICR (p. 22) commends the PCU for M& E and supervision of environmental safeguards.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The PAD (Annex 3) reflected mostly quantitative indicators with a focus on gauging the performance of the project components. M&E activities would be housed within the PCU. This arrangement was designed to ensure follow up of both compliance with component goals and budgetary performance. An end of project impact analysis was included.

Design lacked relevant indicators to gauge the impact of agriculture R&D investments (56% of project costs). Such impact was to be assessed by measuring the improvement in market share of selected production chains in national and international markets which would be difficult to influence in the short term (ICR, p. 5). Design also lacked a specific indicator to gauge the participation of small farmers in the project and there were no indicators to gauge adoption of new technologies at the farm level. Also, M&E design did not include specific indicators to assess the performance of the NBL3Ag laboratory.

b. M&E Implementation:
MADR established a well staffed M&E unit within the PCU early in the project. With Bank support, a Project Management Information System (SIGP) was established to track, monitor and evaluate agriculture R&D sub-projects (ICR, p. 10). A notable shortcoming was the absence of baseline data for the sub-projects implemented by the project (ICR, p. 34).

a. M&E Utilization:
The PCU disseminated major findings and R&D agendas for 12 supply chains through annual workshops and seminars involving relevant sector authorities and stakeholders at the national and regional levels. After delays, the PCU, with Bank support, prepared an impact evaluation study of the Competitive Fund to assess the achievement of the overall project objectives and was utilized in the preparation of the ICR.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
Environment. The project was classified category “B” under OP4.01 Environmental Assessment. The project triggered safeguard policy OP4.09-pest management. OP4.11-Indigenous Peoples- was not triggered although the ICR (p. 10) pointed out that 2% of the small farmers impacted by the project were expected to be indigenous people. An Environmental Assessment identified the need for an Environmental Action Plan (EAP) and a Pest Management Framework. According to the ICR (p. 10) a PMF was prepared, consulted and approved by the Bank. Environmental aspects of the sub-projects were managed according to the EAP. Bank supervision missions included an environmental specialist and sub-projects were closely supervised to ensure environmental compliance. Also, the PCU, except for last few years of the project, included an environmental specialist among staff (ICR, p. 10). The ICR (p, 10) reported that there were no serious environmental problems that arose during project implementation. By project completion 99% of the sub-projects presented socio-environmental assessments of the potential impacts of the technologies developed through project funding and all of these assessments were considered acceptable.

Social. Project preparation included a social assessment that identified the main socioeconomic and cultural characteristics of small farmers across different supply chains. No social safeguard was triggered by the project. The project also included a Small Producer Development Framework (PAD, p. 24) to integrate small farmers in the overall development strategy of the country and in particular the agriculture competitiveness strategy. The ICR (p. 10) pointed out that 2% of the small farmers impacted by the project were expected to be indigenous people.

b. Fiduciary Compliance:
Procurement. According to the Bank Procurement Post Reviews procurement performance up to 2011 was rated satisfactory and risk as average with the exception of a moderately satisfactory rating relating to the procurement processes of the NBL3Ag laboratory. The project procurement load was heavy given the size, diversity and large number of sub-projects. The PCU procurement team was experienced and qualified and there was no evidence of deviation from Bank standards or of misprocurement.

However, two issues were problematic, first, administration of contracts related to the construction of NBL3Ag laboratory; and second, the delayed submission of Statements of Expenditure (SOEs) by sub-projects. The Bank project team worked closely with the procurement specialist, CMU, and PCU to follow up on recommendations.

Financial Management. Financial management performance was moderately satisfactory throughout project implementation. Project risk was assessed as substantial due to project complexity which included country wide implementation of diverse sub-projects by several research entities and universities (ICR, p. 11). Audits and unaudited financial reports were received within contractual dates. External auditors issued unqualified opinion (without exception) on the project financial statements for the audit periods 2008 to 2011 (ICR, p. 11). MADR action plan adequately addressed the auditors recommendations.

However, there is an issue of fiscal nature related to the completion of NBL3Ag being reviewed by the CGR. Also, due to deficiencies on monitoring of sub-projects there is a pending advance in the amount of approximately US$335,000. According to the ICR (p. 11) MADR would reimburse the Bank for any unsupported or unused funds.

c. Unintended Impacts (positive or negative):
The construction of the NBL3Ag laboratory was not explicitly envisioned at project appraisal. However, given the increased risk of Foot and Mouth Disease and its potential impact on the important livestock sector, the project invested a relatively high amount of component 2 resources in the construction of the NBL3Ag laboratory. Delays in the operation of the NBL3Ag cast a negative perception of the overall project performance.

d. Other:
none



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Significant
Concerns on the sustainability of R&D funding and the absence of a systematic approach to transfer technologies generated through sub-projects to farmers. 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR provided a number of valuable lessons emphasized below with some rearrangement by IEG:
  • Operations in the rural sector with a high degree of innovation require an extended horizon, intensive supervision, continuous support for capacity building, and repeat financing to maximize impact, learning and sustainability. Projects such as Colombia's Agricultural Transition require a longer time that greatly exceeds Bank averages to implement and mature. For the success of such projects, it is extremely important to put in place experienced Bank and Borrower project teams with continuity of leadership and a focus on capacity and institution building.
  • For National Sanitary and Phytosanitary Measures System (SPS) interventions to be sustainable, they must have pre-established prioritization criteria and require counterpart funding. The project experience has shown that it is important to establish, during project preparation/appraisal, clear prioritization criteria for the resources within the system and institutions. Furthermore, the activities supporting the SPS system must also have dedicated counterpart funding especially that SPS services are public goods that require a level of public sector support to ensure effective implementation and maintenance.
  • Private sector actors in the agro-industrial sector and producers should not be assumed capable on their own of adopting the technologies generated through sub-project investments in agriculture R&D. Sub-project investments in R&D are necessary to stimulate technology generation and development in the sector, but not sufficient to guarantee the adoption of the technology by agribusinesses and/or farmers, especially if financial resources and lines of credit are unavailable or inaccessible. Smaller scale producers/entrepreneurs in particular lack the organization, commercial records and collateral to support loans from regular banks in order to adopt new technologies requiring initial capital for implementing, and thus, are likely to remain dependent on public sector support for agriculture extension and incentives for adoption of agriculture technologies in the absence of alternative financing strategies.
  • Competitive Funds may not be the best tool for the promotion of smaller scale research organizations in countries with large and diversified agriculture sectors. As evidenced in the project experience, due to the competitive nature of the Calls for Proposals, the sub-projects were concentrated in only a few research institutions that were able to dominate in most requirements and levels of co-financing. Thus, alternative (but complementary) methods of allocating agriculture R&D funds must also be available in order to also promote all types of research organizations and themes.

14. Assessment Recommended?

Yes
Why?
To assess the sustainability of project supported activities and the status of the NBL3Ag laboratory.

15. Comments on Quality of ICR:

The ICR provided a thorough coverage of the project and a candid account of the shortcomings as well several valuable lessons from the project experience. The ICR also included a stakeholder survey and impact evaluation results as part of the completion process. Overall, the ICR was consistent and outcome driven, however, attribution remained a challenge given the gap between the indicators and the PDO itself. IEG also finds that the ICR provided limited details on technologies generated by the sub-projects and the cost of the NBL3Ag laboratory.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions