Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Small Farmer Development Project


  
1. Project Data:   
ICR Review Date Posted:
04/30/2014   
Country:
Argentina
PROJ ID:
P006041
Appraisal
Actual
Project Name:
Small Farmer Development Project
Project Costs(US $M)
 108  139.42
L/C Number:
L4212, L7478
Loan/Credit (US $M)
 120.74  120.56
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  07/17/97
 
 
Closing Date
06/30/2003 06/30/2011
Sector(s):
Irrigation and drainage (25%), General agriculture fishing and forestry sector (25%), General transportation sector (25%), Central government administration (15%), Sub-national government administration (10%)
Theme(s):
Rural services and infrastructure (25% - P) Participation and civic engagement (25% - P) Other rural development (24% - P) Rural policies and institutions (13% - S) Indigenous peoples (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Ebru Karamete
Kristin Hallberg Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
Project development objective stated in the Loan Agreement (p. 9) is " to increase productive and organizational capacity in targeted poor rural communities. The Project also aims to increase emphasis on rural poverty issues in sectoral policies and to improve coordination mechanisms at the national and provincial levels."

Loan Agreement was amended in August 19, 1998 to include an additional PDO: “Financial assistance provided to poor, rural producers in areas declared in flood emergency or disaster”.

Project Appraisal Document statement of objectives is (p. 2): "to increase the productive and organizational capacity in participating poor, rural communities as indicated by: (i) improved production-oriented infrastructure; (ii) increased production of food for family consumption; (iii) diversification of productive activities; (iv) increased value added of production; and (iv) conversion or integration of beneficiary groups into organizations".

As per IEG’s current practice, this Review’s assessment is based upon the formulation of the project objective as in the Loan Agreement.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 08/19/1998

c. Components:
1. Rural Investments (Appraisal Estimate: US$ 93.1 million, Actual Cost: US$ N/A, Additional Financing Estimate US$ 49.58 million, Additional Financing Cost: US$ 43.83 million).
The Social Program for Agriculture, a subsidiary of Secretariat of Agriculture, Livestock, Fisheries and Food, who had experience with small farmers implemented this component. The component had 3 sub-components: (i) Rural Investment Fund (FAIR) provided small grants for demand-based sub-projects; (ii) Support Services provided training, a market information system and a Participation Action Plan and Indigenous Peoples Strategy; and (iii) Management of Rural Investment Activities at the national, provincial and locals levels financed technical assistance and equipment. The original project targeted 9,000 sub-projects covering 30,000 families, and the Additional Financing targeted 3,000 sub-projects and 22,000 families.

2. Strengthening Rural Development Policy (Appraisal Estimate: US$ 7.6 million, Actual Cost: US$ N/A, Additional Financing Estimate US$ 3.93 million, Additional Financing Cost: US$ 2.36 million).
Directorate of Planning and Agricultural Development implemented this component. It had 5 sub-components: (i) National Strategy Development including studies, the Rural Development Committee and a rural development database (ii) Provincial Strategy Development for demand-driven studies and technical assistance; (iii) Training of sub-national and national agencies; (iv) Appropriate Technology Development for studies and competitive research grants; and (v) Institutional Strengthening for the Directorate of Planning and Agricultural Development for equipment and technical assistance.

3. Project Administration, Monitoring and Evaluation (Appraisal Estimate: US$ 2.9 million, Actual Cost: US$ N/A, Additional Financing Estimate US$ 1.49 million, Additional Financing Cost: US$ 1.17 million).
This component financed technical assistance, studies and equipment for project coordination and to conduct M&E at the national and provincial levels.

4. Flood Emergency (Appraisal Estimate: US$ 0 million, Actual Cost: US$ 30.31 million).
This component was added after effectiveness by reallocating US$ 40.0 million mainly from Category 1 to finance grants and consultant services in selected provinces for eligible poor rural families affected by massive flooding in the provinces of Misiones, Santiago del Estero, Corrientes and Santa Fe. The new component basically financed demand-based, small scale sub-projects, similar to those under Component 1, for the recovery of productive capacity within eligible provinces.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs:
Total costs increased from the appraisal estimate of US$ 108.0 million to US$ 139.42 million because additional financing was included in order to scale up the original project activities, to avoid a financing gap, and to maintain momentum while a new Rural Poverty and Growth project was prepared. Loan funds were reallocated many times, initially to cover Government’s request in 1998 for Bank assistance to implement Flood Emergency Program, after that due to accelerating demand for sub-project Grants once the economy recovered, and finally to support government's Drought Emergency Program for small farmers.

Financing:
The original Loan for US$ 75.0 million was fully disbursed. Additional financing of US$ 45.0 million was added in July 2007. By project closing, US$ 120.5 million had been disbursed and US$ 0.27 million was cancelled.

Borrower Contribution:
It was expected that the Borrower would provide US$ 33.0 million (including US$ 8 million beneficiary contribution) under original loan and US$ 11.80 million (including US$ 2.8 million for beneficiary contribution) under the additional financing. The actual disbursements from borrower contribution was only US$ 17.06 million under the original loan and US$ 2.36 million under the additional financing. Disbursements from beneficiary contribution is not available.

Dates:
The original closing date was extended 4 times for a total of 54 months during the life of the project to from December 31, 2003 to June 30, 2011 initially due to financial crisis in the country negatively impacting project implementation for 4 years and later on due to inclusion of the Additional Financing that was approved on July 17, 2007 for a 4-year period.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial
The project development objectives were relevant to the country priorities. In the mid-1990s, around the time of project appraisal, Argentina’s agricultural sector was undergoing a restructuring program with the goal of increasing comparative advantage through output and export growth. In order to decrease government's role in the agricultural sector, measures such as reducing crop price supports and protective tariffs, forced the sector to diversify production and increase productivity to remain competitive. However, the rural poor, particularly the small scale farmers, who had been traditionally overlooked, lacked means to be able to adjust to these reforms. So, the project objectives were directly tackling these issues of the rural poor.

The project development objectives were relevant to the government's rural development strategy of 1996. The strategy had identified that in order to increase income opportunities for the rural poor and small scale producers, development of community organizations to provide input supply and marketing services; and provision of better extension services and rehabilitation of rural infrastructure were essential.

The project development objectives were also relevant to the Bank 1998-2000 Country Assistance Strategy (CAS), the pillars of enhanced social development including poverty alleviation and human resources development; and improved performance and institutional capacity of Government, especially provincial, to deliver social infrastructure and environmental services. The project objectives were also relevant to 2010-2012 Country Partnership Study, specifically both Pillar I - Sustainable Growth with Equity and Pillar II- Social Inclusion.

One shortcoming was that the additional third objective was expressed in terms of outputs rather than outcomes.

b. Relevance of Design:
Substantial

The PDO and components were reasonably linked: productive and organizational capacity increase for the poor communities was to be achieved through provision of small grants and training activities under Component 1 (and under Component 4 for the added PDO focusing on areas declared in flood emergency or disaster); and increasing the emphasis on rural poverty issues in sectoral policies was going to be achieved through Component 2 activities. However, one shortcoming was that the integration of policy makers at the national and provincial levels with the implementing experts at the field level and beneficiaries could not be achieved with the project design; this could have been achieved if the federal and provincial governments were given larger roles and resources in project implementation through the project design (ICR p. 15).


4. Achievement of Objectives (Efficacy) :

There are three objectives: (i) increase productive and organizational capacity in targeted poor rural communities; (ii) increase emphasis on rural poverty issues in sectoral policies and to improve coordination mechanisms at the national and provincial levels; (iii) financial assistance provided to poor, rural producers in areas declared in flood emergency or disaster.

1. Increase productive and organizational capacity in targeted poor rural communities. Rated Substantial

The project results are based on 2 surveys: the main one being CRISOL 2010, which is the impact assessment of the project until 2010, with a random sample which covered 377 respondents from 5 provinces; as well as FAO 2011 evaluation, which is an economic impact assessment based on 268 respondents. CRISOL 2010 assessment did not include non-beneficiary groups in the assessment or otherwise accounted for other factors that may have influenced the outcomes outside of the project. Although FAO 2011 assessment included control group analysis of without project beneficiaries, it is not clear if and how the random sampling was made.

Outputs:
  • 12,047 sub-projects were financed (exceeding the aggregate target of 12,000) benefiting 70,527 beneficiary families in participating, poor rural communities (target: 52,000 families). Vulnerable populations (women, youth and indigenous people) represented 65 % and 45 % of families under original project and additional finance respectively.
  • On-farm and community infrastructure sub-projects (wells, small-scale irrigation and dams, fencing, livestock and equipment, sheds, mills, machinery and tools, housing improvements) represented 80% of total investments.
  • Targeted communities, private and public service providers were trained on various topics. Trainings were demand driven where appropriate.
  • A market information system was to be installed in each province to provide on-line information in markets, but no evidence provided on its implementation.


Outcomes:

The ICR presented the following information on outcomes:
  • Volume of production increased for 45% of agricultural producers. While the production and yields of specific food crops was not reported, production of agricultural crops overall rose 15.3% and 19.2%, and of livestock products 20% and 23.3% under the original loan and additional financing, respectively (during project period).
  • 89% of all beneficiaries of infrastructure investments increased their capital assets.
  • 56% of farmers benefiting from infrastructure projects increased their production and quality; 71% of livestock producers increased productivity and 60% improved quality.
  • 33% of agriculture producers specifically- diversified production and improved the conservation of their product.
  • 56% of project beneficiaries increased their net farm income during project period.
  • The proportion of farmers with the lowest registered income pre-project decreased from 15% to 5% indicating important poverty reduction effects.
  • Producers were organized under 1,550 associations by either the conversion/upgrading of existing organizations or creation of new ones. Approximately 40% of beneficiaries belonged to an association. Over 60% of farmers conducted joint purchasing of goods and services and some 20% were marketing jointly, still organized marketing arrangements was challenging for many small farmers, as they were still subsistence farmers. However, there was no marketing strategy developed for the sub-projects.

It is important to note that the ICR did not provide any comparative data on non-beneficiary performance, or otherwise control for other factors that may have affected the outcomes (such as other ongoing projects or policies affecting these same regions, agricultural price developments, weather developments etc.). The project team subsequently explained that it was intended to establish control groups but because beneficiary profiles had to be redesigned during the course of this long project, the sets of profiles ended up not being comparable and the surveyed group not being statistically significant enough to be compared to a control group. Although the CRISOL evaluation was not able to compare beneficiaries with a control group, it did use a variety of methods including focus groups appropriate to CDD projects. Considering that the project led to an increased organizational and productive capacity as well as reached out a sizeable number of small farmers to finance sub-projects by exceeding the targets. the achievement of this objective is rated Substantial, but only marginally noting the weakness of attribution.

2. Increase emphasis on rural poverty issues in sectoral policies and to improve coordination mechanisms at the national and provincial levels. Rated Modest.

Outputs:

  • A national rural development strategy was developed. 19 of the 23 participating provinces prepared a rural development strategy .Only 15 provinces dedicated concerns regarding small farmers within rural development strategies.
  • The project financed 23 studies on rural development issues and 21 publications on rural poverty and small farmers as the foundation for strategy and policy development.
  • An aggregate 56 adaptive technology research projects for small farms were implemented and disseminated through online catalogues and special events, giving increased visibility to diverse institutions and organizations working with this theme.
  • A National Register of Family Agriculture was created, using baseline data developed for the Mid-term Review; the goal is to register every producer. A sub-component for Strengthening Family Agriculture Organizations was incorporated into the Additional Financing, resulting in the financing of 30 proposals from small farmer organizations for specialized technical assistance for capacity-building.
  • Successes and the results of key studies were disseminated as critical input for the national and provincial strategy and policy dialogue.
  • Many provinces (exact number was not provided by the ICR) reorganized key institutions to improve rural development management and coordination and established specialized institutional units (sub-secretariats, institutes, departments) dedicated to improving contacts with and responsiveness to small farmers and rural workers.
  • Provinces also updated their rural diagnoses in many cases including new databases, conducted activities to improve links to municipal/local governments, and participated in project sponsored training and capacity-building for rural technicians to improve socio-organizational and productive skills.
  • A sub-secretariat for Family Agriculture under the Rural Development Secretariat, and National Family Agriculture Forum were established as an opportunity for dialogue between the Ministry and organizations of family farmers, and to speak on behalf of small farmers, and the subsequent formation of provincial branches of this organization.

Outcomes:
The ICR presented output data on this sub-objective, rather than outcome data. This was also because outcome indicators were not very linked with the sub-objective and mostly measured output. The ICR reported that over time, 19 provincial rural development strategies were increasingly based on small farmer characteristics, diagnoses and databases, and on direct consultation. However, sub-projects financed under component 1, tended not to be anchored in the local rural development strategies. Also a more strategic approach was needed linking the sub-project financing program with the institutional strengthening activities. Furthermore, the ICR reported that (p.15) provincial adhesion to this sub-objective was variable. The project team subsequently stated that there was increased attention to rural poverty issues, and more professionals were being trained in the problems of family agriculture. Also, recent feedback coming from the field via other Bank projects confirm that provincial rural development authorities and extension technicians are working together with the professionals of the Secretariat of Family Agriculture indicating that coordination of rural development policies and programs have been progressing. Based on that , this sub-objective is rated Modest.

3. Financial assistance provided to poor, rural producers in areas declared in flood emergency or disaster. Rated Substantial

Outputs
The Flood Emergency Program disbursed about US$24.31 million - of the US$40.0 million reallocated for this purpose and 39,322 beneficiary families were assisted compared to a target of 30.000 families in 4 provinces. Although the funds for this sub-component could not be utilized completely, the number of families reached exceeded the target.

Outcome
According to the survey that covered the four provinces, farmers who did not receive project support were in many cases forced to migrate; and beneficiaries rated the financial support highly for enabling them to respond to basic flood-related challenges.

5. Efficiency:

Modest

During the appraisal stage Internal Economic Rate of Return (IERR) was calculated for 9 sub-projects that covered 5 geographic regions and based on productive systems commonly found in these agro-ecological zones. Price and technical coefficients were provided by the government technical agents in the respective provinces. Detailed surveys to portray the baseline situation of the beneficiaries were also carried out. Estimated IERRs for the model sub-projects ranged from 14% to 36%. Considering the projected annual number of sub-projects, an average IERR of 26.3% for all types of sub-projects was estimated under the Rural Investment component. An average IERR of 19.0% was calculated for the overall project when the costs of activities were included. The details of the survey data on which the analysis was based, the main assumptions for estimated benefits as well as assumptions to come up with the overall IERR (with the exception of discount rate, duration and prices) were not provided by the PAD, so validity of these ex-ante results could not be verified.

The ICR presented the results of an economic analysis that was carried out by FAO in 2011. However, the methodological differences between the two stages (ex-ante and ex-post) were not provided. According to the ICR, the beneficiary population was compared to a similar population group used as a control group and the analysis used an impact assessment of the first stage project conducted by CRISOL in order to obtain the characteristics and indicators of the population as of end of 2010. A sample of 268 families (out of the total of 70,527) divided among all project provinces were included. Baseline information including cropping area, livestock numbers, employment and family income was used to construct without project situation. The analysis used two methods:

(i) Reflexive comparisons: changes in indicators during the project, i.e. before and after intervention effects and;

(ii) Cost-benefit analysis of discounted flow of funds using sample data to construct farm models and obtain rates of return (with 12 % discount rate and for a duration of 20 years). 12 farm models that represent the majority of cases and were used as examples of the most common typologies. Since the Crisol database did not include data on yields, incomes and costs of production per hectare, provincial regional authorities involved in the Project provided this information. Constant 2011 prices were used and cost of labor (market prices) were included only in the economic analysis. Gross value and cost of production for without project and with project situation was calculated for year 12, and this served as the annual incremental economic income for the original project. Second stage (additional financing) was also similar: with and without project results of year 7 were used to calculate annual incremental economic income.

According to the before and after effects results, family income for total sampled families of 268 increased by 136% and capital invested by aggregate sampled families rose by 206%. Although, there was inclusion of non-project beneficiaries within the surveyed sample, no control group comparison was provided for these results.

The cost benefit analysis showed high internal rates of returns: the IERR for the original project was 129 % and IERR for the additional financing was 108%. According to the ICR, the reason for high IERRs was (p. 17): "...the value of the public investment, financed in major part by the World Bank, was much smaller in relation to the net income received by the beneficiaries in the without-project situation". The project team subsequently clarified this statement by stating that the high returns were a result of wrong estimations (during preparation) of the ratio of production value to investments. In other words, although the project exhibits good economic results it is possible that the EIRRs have been over-estimated. The team subsequently recalculated the ERR by changing the assumptions. Accordingly, the ex-post calculation had included all subsidies as income of the investment, and recalculating the ex-post IERR without including project subsidies as part of the income flow, resulted a markedly reduced Economic Rate of Return, of 69 % for both stages of the Project.

In terms of operational and administrative efficiency, the project closing date was extended numerous times totaling 54 months (4.5 years). The reason for this very long extension was partially the economic crisis with its concussions and national recovery during 2000-2004 as well as preparation of the additional financing. However, there were administrative issues as well: project continuity and coherence were negatively affected by repeated, turnover of top management within both national-level and provincial sector institutions. Also, the Additional Financing became effective two years after closure of the original loan and 18 months after Board approval, which seriously disrupted the continuity of the project.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

According to ICR guidelines, the combined outcome rating of a formally restructured project is weighted according to the proportion of the Credit that was disbursed before and after the restructuring. However, since the third objective was added in 1998, soon after project effectiveness, and all of the disbursements occurred after the restructuring, 100 % of the weight is given to the ratings under the revised objectives, i.e., all three objectives.


Relevance of objectives is rated substantial due to relevance with country priorities and strategies; relevance of design is rated substantial due to the reasonable link with the PDO and the project components; objective 1 is rated substantial only marginally noting attribution difficulties; objective 2 is rated modest due to lack of sufficient evidence; and objective 3 is rated substantial; the efficiency is rated modest due to administrative and operational inefficiencies.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

This review’s assessment of risk is the same as the ICR’s, which is ‘moderate’. . ICR reported that (p.20) , there were adequate O&M arrangements set up for most projects and sustainability was elevated through functioning community organizations and social capital. Although the ICR stated that the investments lacked marketing aspect and there was insufficient technical support for beneficiaries (ICR p. 29), the team subsequently provided the following information:

"Beneficiaries surveyed valued project-financed technical assistance as the most important aspect of the project. Under the proposed follow-on project, all technical assistance was to be provided by the Borrower because the staff was already in place (having gained experience under the project). Extension staff contracted at the provincial level for the Additional Financing became regular staff of the relevant Rural Development agencies, providing strong support for both ownership and sustainability. During and after the project’s Additional Financing, the Borrower agency hired project technical/other personnel using own budget resources. This staff is still in place today, collaborating with provincial authorities in rural development programs. In terms of marketing 31% of beneficiary farmers started to market their surplus production while another 22% increased existing sales. About 28% of farmers said they had started to market their product as a direct result of the project. More than 50% of all livestock producers increased sales/marketing due to higher production stimulated by the project. Also participation in an association was found to be critical to getting the full benefits of marketing especially in regard to price negotiations (41.5% of farmers in associations had negotiated better prices vs. 18.3% of farmers not participating) and actually starting to sell (35% vs. 26%)".

All of these lead to Moderate risk to development outcome rating.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
The project’s sub-project investment activities were developed on the basis of a household survey carried out during preparation. Although this social underpinning of the project was comprehensive, there was no strategy for targeting vulnerable populations such as women and youth by the project. Although the PDO did not mention targeting specific populations such as women and youth, project design documents suggested that these special populations were to be targeted (ICR p. 5, para. 2.1.3). The Additional Financing was mainly based on the experience and lessons of the original project, but did not resolve the issue of women and youth targeting. Preparation was done through a participatory approach with numerous consultations, workshops, focus groups and interviews that obtained stakeholder input in project design.

Project design drew from the lessons of rural development programs of 1960-1980s and social investment fund of the 1990s. Indeed, Secretariat of Agriculture, Livestock, Fisheries and Food created the Social Program for Agriculture to pilot a decentralized, demand-based program for poor farmers in 1990s. The main lessons were the need for: transparent and objective targeting criteria; and technical assistance and Operation and Maintenance systems built into sub-project preparation to ensure sustainability.

Institutional arrangements were complex. Components 1 and 2 were designed as two separate operations implemented by two separate directorates (Secretariat of Agriculture, Livestock, Fisheries and Food's subsidiary, the Social Program for Agriculture was responsible from implementing Component 1 and Institutional Strengthening for the Directorate of Planning and Agricultural Development was responsible from managing Component 2). The project lacked mechanisms for integrating the two components to achieve synergies.

As mentioned in Section 3b, results framework had major issues in terms of unclear objectives and delinked and unclear outcome indicators. Monitoring and evaluation design also had several issues including operational data not being organized well: difficult to retrieve and use for results-based management or for evaluation (please see section 10).

Risk identification and mitigation at appraisal missed the following: (i) There were wide disparities in provincial capacity to engage with and/or implement strategic goals for rural development and rural poverty reduction, which could be addressed through adequate incentives for the provincial governments. (ii) National coverage of the project in very remote locations in a very large country, although had significant implications in terms of oversight and fiduciary, was not one of the risks.

The ICR reported that (p. 21), a Quality at Entry Assessment done during appraisal rated all aspects Satisfactory/Highly Satisfactory except readiness for implementation that was rated Marginal due to inadequate realism and quality of the implementation plan for the first year. The QALP-1 in 2008 downgraded quality at entry to Moderately Satisfactory due to an imprecise PDO and weak causal links to the indicators, inadequate arrangements for M&E and lack of baseline data, a need for attention to the local-level transparency of procurement decisions and flow of funds (reflecting new Bank fiduciary rules for CDD projects), questionable Borrower commitment, and a lack of specific approaches for including vulnerable groups despite the strong social analysis.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
As reported by the ICR, (p.21), there were regular supervision missions (30 missions in 13 years) with action plans with a strong relationship with the Secretariat of Agriculture, Livestock, Fisheries and Food ; the team had a rapid response to borrower's request for flood emergency support. However, there was also: failure to modernize the results framework; inadequate attention to M&E; inadequate attention to environmental compliance throughout the original project, and unrealistically positive Implementation Supervision Report ratings during critical periods under the original project.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Based on the evidence provided by the ICR the rating leads to moderately unsatisfactory. As reported by the ICR, (p.22), the government was supportive of addressing poor rural populations' issues, as well as decentralized demand driven programs and autonomy for provinces to design their own rural strategies (despite political reservations). However, the prolonged crisis between 2000-2004put the project on the spot. Also, weaknesses in national and provincial government commitment to the project made implementation difficult. The long delay of 18 months for effectiveness of the Additional Financing reduced the number of investments. The Borrower did not prepare a Borrower Completion Report, although it helped with the final impact assessment.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
The Social Program for Agriculture and the Secretariat of Agriculture, Livestock, Fisheries and Food implemented the first decentralized and demand-driven rural investment program of national scopein Argentina. The agency was not influenced politically, and maintained project's ethical profile in the provinces and field. However, Social Program for Agriculture and the Secretariat of Agriculture, Livestock, Fisheries and Food top management had frequent turnover that negatively affected project management.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
Key performance indicators supporting the project development objective lacked targets, their measurability and link to the objective was questionable, and along with numerous intermediate outcome indicators represented a difficult monitoring task. The emphasis was on outputs not outcomes. Attempts by the Additional Financing to improve this framework added to the monitoring burden by increasing the number indicators but did not address the real issues. An MIS system was established, and a baseline study was done in 2002.

b. M&E Implementation:
Fiduciary monitoring and reporting functioned well but operational data was not organized adequately: it was difficult to retrieve and use for results-based management or for evaluation and final reporting. In terms of products, the project financed a Baseline Study (SAGPyA, 2002), two impact monitoring studies (SAGPyA, 2001 and 2005), a Mid-term Review evaluation (FAO, 2002), and a final evaluation of the original project (Crisol, 2010). The baseline did not use a formal survey based methodology;information was collected by project technical staff from beneficiaries as they entered the program, i.e. a detailed farm profile at entry. There was no physical performance study of component 1 sub-projects. An exhaustive physical performance study could have been useful to draw lessons for future demand driven projects, but for the purposes of this project the impact survey was sufficient.

a. M&E Utilization:
Workshops and seminars and web-based dissemination was used to disseminate project results. Catalogues of project-financed adaptive technologies were disseminated to over 120 public and private institutions and civil society bodies but the extent to which these technologies reached farmers during the life of the project is not known. No other information on utilization of M&E data was provided by the ICR.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The Project was Category B in terms of environmental safeguards. During appraisal, environmental analysis of the project was carried out and an Environmental Management Framework (EMF) was prepared and measures were integrated into the sub-project screening, evaluation, approval, and monitoring procedures. The National Coordination Unit hired an environmental specialist but ICR reported (p. 11) that there was very little follow up. Also, according to the ICR, no report of safeguards took place until 2010.Under the Additional Financing, Bank safeguards specialists noted satisfactory attention to environmental sustainability in FAIR sub-projects.

A comprehensive social assessment was completed as part of project preparation to address issues regarding poor households and potential types of activities to be supported. The project was approved in 1997 before the new indigenous peoples and resettlement policies came into effect. During appraisal a diagnostic of the indigenous population was prepared and community workshops were held with indigenous groups to incorporate their needs into the design of the project. A strategy for indigenous peoples in the provinces of Salta and Formosa was prepared. Indigeneous peoples represented 10-12 % of total beneficiary families under the project. Additional targeting criteria were applied to reach denser concentrations in certain municipalities. The Bank mission in 2010 reported that the project was in compliance with OP 4.10 (Indigenous Peoples) but also noted that record-keeping on Indigenous Peoples consultations and sub-project preparation needed improvement.

b. Fiduciary Compliance:
Fiduciary performance was rated Moderately Satisfactory by the QALP-1 in 2008. Audit reports showed unqualified opinions throughout the project’s life. The review of the 2006 audit report found problems with internal control procedures at the sub-national level in two provinces at which point the FM rating was downgraded to Moderately Unsatisfactory. Following that an Action Plan was agreed with the National Coordination Unit to resolve identified weaknesses, which were mainly administrative errors and lack of knowledge. FM supervision rated the National Coordination Unit's financial administration Moderately Satisfactory in October 2009 based on compliance with the Loan Agreement and use of project funds for intended purposes. This rating was sustained through closing.

In terms of procurement, the QALP-1 in 2008 detected no significant problems in procurement provisions or practices. Two procurement post-reviews of the Additional Financing found no major issues and rated procurement Satisfactory. All Implementation Supervision Reports rated procurement satisfactory with one exception in 2002.
ry with one exception in 2002.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
An additional Financing instrument, with its maximum 3-year period, may not be a good option in countries with generic delays in loan effectiveness. Such loans still need to go through the same bureaucratic procedures in-country as a regular investment loan. Bureaucratic delays affecting the signing and effectiveness of Additional Financing can cut the implementation time by 50%. A separate, follow-on operation, with preparation started early and with a standard life-span, may be the better option.


Effective oversight of projects with national scope can be logistically and financially challenging particularly in countries with vast territory and projects implemented in remote places. This project showed that more selective, regionally-focused projects could make better sense in countries like Argentina.

Mechanisms need to be developed for engaging provincial authorities in rural poverty issues and build their commitment. Greater coordination and cooperation between field teams and provincial and municipal institutions and authoritiesis also important to achieve decentralized decision making. Federal/provincial agreements - via activities which give the provinces a major role in achieving project objectives, backed by appropriate financing -- may be helpful. Creating synergies, disseminating information, and building participatory networks for direct interface between policy and field practitionersand beneficiaries are essential.

Mid-Term Reviews and Additional Financing should be used to address project design
weaknesses and ensure that M&E arrangements are adequate. Inadequacies in the initial design and organization of the results framework and M&E arrangements should be corrected during Mid Term Review or Additional Financing. Teams need to ensure that evaluation studies and the MIS are building an information base consistent with the design framework/indicators, by end-project. An independent physical performance study conducted around mid-term is recommended for CDD operations, especially those with broad geographic coverage.

This review adds the following lesson:

Although cancellation would have limited the knowledge gained from implementing Argentina’s first CDD operation, extending the project for many years without addressing design issues was not a very efficient way of using Bank resources. The shortcomings in the M&E system (unaligned and unmeasurable indicators), should have been addressed early on, without waiting for the MTR or the additional financing. The issues regarding Component 2 design should have been resolved through a restructuring as well.


14. Assessment Recommended?

Yes
Why?
Project is unique as the first CDD operation in Argentina and one of the longest operations in the region. As a national program, it reached considerable number of beneficiaries, however the evidence presented by the ICR was not sufficient to conclude substantial achievement of objectives and efficiency. As the ratings differ from the ICR's, it is worth making a detailed analysis in the field.

15. Comments on Quality of ICR:

ICR was comprehensive with candid report of implementation description and challenges. However it lacked detailed description of surveys which the outcome and economic analysis data was based upon.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions