Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Sd - Support To Agriculture And Forestry Development Project (erl)


  
1. Project Data:   
ICR Review Date Posted:
04/30/2014   
Country:
South Sudan
PROJ ID:
P104786
Appraisal
Actual
Project Name:
Sd - Support To Agriculture And Forestry Development Project (erl)
Project Costs(US $M)
 42.90  31.97
L/C Number:
Loan/Credit (US $M)
 25.80  30.17
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  11/06/2007
 
 
Closing Date
10/31/2011 12/31/2012
Sector(s):
Central government administration (50%), Agricultural extension and research (30%), Forestry (20%)
Theme(s):
Rural policies and institutions (67% - P) Rural services and infrastructure (33% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Ebru Karamete
Lauren Kelly Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
The Project Development Objective, stated in the Project Appraisal Document- Final Proposal for a Multi-Donor Trust Fund Grant (p.6), is "to increase productivity of some targeted 730 agricultural groups and 260 forestry producer groups by: (i) facilitating adoption of improved technologies in agriculture and forestry; through a matching grant mechanism to be implemented by experienced NGOs and, (ii) strengthening capacities of the Central and State Governments and the private sector to plan and respond to identified needs in agriculture and forestry ".


The Grant Agreement statement of Project Development Objective is (p. 4): "to increase productivity of targeted agricultural and forestry producers, by: (i) facilitating adoption of improved technologies in agriculture and forestry; and (ii) strengthening capacities of the GoSS, State and Local Governments as well as private sector to plan and deliver services in agriculture and forestry".

The two statements are effectively identical but as per IEG’s current practice, this Review’s assessment is based upon the formulation of the project objective as in the Grant Agreement.

The project was restructured, with VP approval, with the following revised objectives: "to increase the productivity and production of participating smallholder farmers in agriculture and forestry" (p. 3 of Amendment to the Grant Agreement). Since, there was no Board approval, this review considers this adjustment to be a Type II restructuring and therefore assesses the project against the original objective (For details see Section 2.d).

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? No

Date of Board Approval: 03/23/2011

c. Components:
The project was originally planned to be implemented in two phases: the first phase of two years would focus on rapid technology dissemination to beneficiaries; while the second phase, of three years, would mainly support institutional development. However, this phased approach was discontinued eventually due to delays in project implementation.

A. Establishment of Support Services (Appraisal Estimate: US$ 25.0 million, Actual Cost: US$ 17.20 million).
The Component included 3 sub-components: (i) establishing group-based advisory services and financing through a matching grants scheme; and training of public and private sector extension providers. The activities were facilitated by a Lead NGO and Local Level Organizations; (ii) setting up of forestry support services and the promotion of community forestry management, including community-based investment projects; training of public and private sector service providers; and formulation of a forestry sector strategic management framework; (iii) technology development and transfer, that included strengthening of applied research in crops, agro-forestry and livestock with a focus on technologies for improved food security; and support to agricultural and forestry research system through the development of Research Management Framework. During the project restructuring agricultural and forestry support services were merged due to the reported limited demand for forestry support services by beneficiaries. According to the ICR, the beneficiaries reported a preference for crop production over tree-planting.

B. Institutional Development (Appraisal Estimate: US$ 17.90 million, Actual Cost: US$ 12.90 million).
The component aimed to strengthen central and state government planning and capacity development in agriculture and forestry. Support provided on policy analysis and formulation, defining MAF’s organization and management procedures and the development of an institutional framework for rural development. A Monitoring & Evaluation System and a Management Information System was planned to be established under Ministry of Agriculture and Forestry (MAF). The project financed training as well as refurbishing and/or construction of infrastructure for the Ministry including training centers and research stations, mobility improvement, office equipment and material provision at the state level.

C. Support to Project Implementation (Appraisal Estimate: US$ 0, Actual Cost: US$ 1.90 million).
This component was added following the Mid Term Review and provided support MAF to strengthen project coordination and implementation.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost
Total project cost at appraisal was estimated at US$ 43 million. Actual costs were US$32m or 75% of the appraisal estimate due to the decline in Borrower’s actual contribution as a source of financing.

Financing
The project was funded by a multi donor trust fund (MTDF) that was developed to provide funding to reconstruction and development needs of Southern Sudan, financing priority projects and programs that were both pro-peace and pro-poor. The original amount of funding allocated by the MDTF for this project was US$ 25.8 million. The finance was planned to be utilized over two phases: Phase I would utilize US$ 9.8 million of the total. In December 2009, the funding amount was increased to US$16.17 and the two phase approach was discontinued. A bilateral donor – the Netherlands, provided an additional US$14m to the MDTF in March 2011 to support the implementation of this project. By project closed, the project had utilized a total of US$30.17 million of MDTF finance.

Borrower Contribution
At appraisal borrower contribution was estimated at US$ 17.1 million (US$8 million for Phase I; US$9.1m for Phase II). Actual Borrower contribution was US$1.8. The Borrower indicated that this reduced contribution was due to a decline in revenues owing to a fall in the price of oil. Although the Borrower indicated that it could increase its contribution after the crisis was managed, this never came to fruition.

Dates (including Type II Restructuring). The project was approved in August 2007 and was designed to be implemented over a 5 year period, with work carried out in two phases (Phase 1 mid-’07 to mid-’09; Phase 2 mid-’09 to mid’11). The project closing date was extended three times for a total of 14 months from the original closing date of Oct 31, 2011 to Dec 31, 2012. The project indicated that more time was needed to complete planned activities, owing to financial shortfalls from the planned Government contribution and implementation challenges (eg. Heavy rains had damaged some of the road networks and impeded progress along the way).

Type II Restructuring. As part of the additional financing arrangement on March 23, 2011, the project underwent a modest revision in the formulation of the PDO which was approved by the Oversight Committee of the MDTF and the Bank’s regional VP. The revision was made to align the PDO with the new financing agreement. Indicators and targets were slightly revised to simplify collection of data and introduce clarity in measurement.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial. The objective of increasing agricultural productivity was and remains substantially relevant in South Sudan, where a high reliance on food imports have contributed to severe food shortages. The civil war left the country with widespread displacement of population, difficult humanitarian conditions and food insecurity and malnutrition as well as very limited infrastructure. As a result of large pieces of agricultural land that was left uncultivated, agricultural production was inadequate to meet the demand for food. The Interim Strategy Note described the agricultural sector by stating (p. 9): "About 63 percent of the total land area in the country is covered by trees and shrubs, while cropland accounts for merely 4 percent. The areas with high to medium agricultural potential due to favorable natural conditions for crop production account for 53 percent of total land area. Reaching the level of average crop yields observed in neighboring East African countries could easily triple agricultural production." A UN Food and Agriculture Agency (FAO) satellite land cover survey conducted in 2012 revealed that just 4.5 percent of South Sudan’s available land was under cultivation; as of 2012 the World Food Program estimated that roughly one-third of the country required food aid.

However there are several ways in which this objective could have been modified to increase its relevance. The project objective should have included capacity building as a main aim, rather than an embedded and phased one. It should have been simplified to address crop production needs rather than both agriculture and forestry. The project would later reveal that agricultural crops were preferred over tree crops in the immediate transition period. With an average smallholder farm size of 0.2-3 ha, tree cops competed for scarce land space and water, and harvesting would require years, not one or two seasons. The objective includes quantitative targets for the number of farmer groups that would be supported under the project, while disclosing that due to the emergency nature of this project, no prior diagnostics were conducted, so that it is not clear how the project arrived at this target. (The quantifiable targets were later removed in an type II restructuring of the project towards project close).

The World Bank did not have a country strategy in place during the MDTF period, implemented under the Transition Government. Rather, actions were guided by activities planned under the Comprehensive Peace Agreement. A World Bank Interim Strategy Note developed to learn lessons mid-way through the MDTF experience found that [MDTF] activities had been challenged by excessive expectations raised by the Comprehensive Peace Agreement, donors’ unrealistic expectations of government capacity, and an underestimation of the challenges that would be faced in implementation. The World Bank’s current Strategy – an Interim Strategy Note for FY13-FY14 places agriculture at the heart of enhanced economic and social well-being, but does not include forestry. South Sudan Development Plan (2011-2013) also includes increased agriculture production as one of its economic development goals.

b. Relevance of Design:
Modest. The major challenge experienced by this project design was the bifurcation of its implementation strategy. The project objective was designed to be achieved through two means (1) extension of improved technologies in agriculture and forestry sector and (2) capacity building of the National, State and private sector service providers. Both were deemed necessary to achieve increased productivity aims. However project design staggered these activities, so that the extension services were implemented a priori during the first phase, while critical capacity building activities were left for the second. The project was designed in an environment of funding insecurity with regard to the MDTF, Government contributions and donor expectations. The decision to phase the project approach in this environment introduced a significant level of risk of non-achievement. The bifurcation also allowed the project to heavily rely on the lead NGO and community facilitators to organize producers and to deliver services without simultaneously building support for service delivery capacity within the local and state governments. The original design did not envisage a component for supporting project implementation – since the lead NGO would implement the first phase. But by not anchoring the first phase in state and local government functions, the project missed an opportunity to facilitate a smooth transition between service delivery models and to sustain investments made in the project supported network of community facilitators.


4. Achievement of Objectives (Efficacy) :

The project development objective is “to increase productivity of some targeted 730 agricultural groups and 260 forestry producer groups by: (i) facilitating adoption of improved technologies in agriculture and forestry; through a matching grant mechanism to be implemented by experienced NGOs and, (ii) strengthening capacities of the Central and State Governments and the private sector to plan and respond to identified needs in agriculture and forestry.”

The achievement of the project objective is rated modest, mainly owing to the lack of attributable evidence on agricultural and forestry productivity in the project areas. As referenced in the Section on Relevance of Objectives, the objective of increasing productivity in two major sectors of the South Sudan’s economy through a trust-funded activity in a very time bound activity cycle was overly ambitious. Measurement challenges – associated with the specificity of the objective - could have been anticipated. This project produced several noteworthy outputs however are highlighted below.

Outputs:

(i) Facilitating adoption of improved technologies in agriculture and forestry.

1. The project introduced an advisory service model for agriculture and forestry using group based advisory services
2. 777 farming groups (total membership 15,370 and 40 % women) were organized, although this fell below the intermediate outcome target of 990 farmer groups
3. 115 micro-projects and 10 Special Projects were implemented to generate new technologies for distribution – this included seeds, fruit nurseries, forest nurseries. A reported 334 tons of quality seed was produced under the project.
4. Basic agricultural technologies were transferred to beneficiaries, including improved varieties for sorghum, maize, groundnut and cassava, fruit and vegetables.
5. Improved production techniques were introduced such as planting in rows, appropriate spacing, timely planting and weeding.

(ii) Strengthening capacities of the State and Local Governments as well as private sector to plan and deliver services in agriculture and forestry.

1. A total of 1234 staff of Ministry of Agriculture and Forestry were trained. There is no other information provided on the effectiveness of this training, or the uptake
2. In total 16 various government facilities were reconstructed/refurbished, including 3 zonal research stations.
3. 30 vehicles, 34 motorcycles and various other equipment were distributed to states/institutions.
4. 5 thematic agricultural sector policies were prepared.

Outcomes:

As expected in the short-time frame allowed for the implementation of these activities, the project does not provide sufficient evidence that these practices were adopted technically or culturally or that the technologies were appropriately adapted to the agro-ecological context. The project provides evidence that the technologies led to land extensification, which in a country characterized by massive internal displacement, should have been accompanied by an explanation of the land situation in the project areas. For example, the project reports that the area under staple crop cultivation for beneficiaries increased from 1 feddan per household on average to 2.27 feddan per household. Survey data on yields- showing crop intensification - was included in the ICR, but it does not reflect yields attributable to the project, and project affected yields cannot be discerned with the data made available.

An agricultural survey was conducted by the Ministry of Agriculture during the project period, but the area surveyed mixes project and non-project areas. Even so, there is no information available to validate the quality of the survey data (methodology, sample size, beneficiary identification). It is also not clear whether the assessment considered exogenous effects, such as rainfall or increased labor after the war, since the methodology for the survey was not provided by the ICR. The data provided by the ICR, which is not limited to the project intervention, is included below. In an environment marked by a return from war, increased yields would be expected given the low or negligible baseline that could be anticipated. However, the project invested some US$17m in technology development and transfer, that included strengthening of applied research in crops, agro-forestry and livestock with a focus on technologies for improved food security. Evidence is lacking that the project investment yielded adoption of the project supported technologies, and that the adoption has increased yields.

Crop Yield Data from the ICR

1. Sorghum yields increased from a baseline of 0.2 tons/feddan to 0.48 tons/feddan
2. Maize yield increased from the baseline of 0.25 tons/feddan to 0.55 tons/feddan
3. Groundnut yields increased from the baseline of 0.35 tons/feddan to 0.61 tons/feddan
4. Also, some 138,066 tree seedlings raised, planted or sold by farmers

The ICR does not provide any evidence that the productivity of the 360 targeted forestry producer groups was increased. It does not provide data on livestock, agro-forestry, or forestry yields, even though these were some of the main technologies transferred. This is relevant since states like Upper Nile, Unity and Jonglei are highly dependent on livestock, whereas East and Central Equatoria are more prone to mixed use.

5. Efficiency:

Efficiency is rated modest since neither the PAD nor the ICR presented an economic rate of return analysis, or a cost-benefit analysis, due to lack of data and a weak project M&E system. According to the ICR, the project did not collect farm management data that could enable the computation of farm incomes and no information is available regarding land clearing and the use of farm inputs and their costs, thus precluding any useful undertaking in economic and financial analysis. The only information collected consistently by the project was crop yields and the number of farmer groups and their members. An attempt by the project to measure incremental yields – using maize data as a proxy for all “other crops” may point to production gains, but not efficiency gains. Without information on inputs and costs, an accurate assessment of the efficiency of this project, or a value for money analysis, cannot be undertaken. The project could have attempted to collect data on farm inputs, costs and yields, with regard to the pre and post project scenario using recall data (with and without the improved technology). The project could have also contributed information on the cost of service delivery vis a vis the NGO model that was subscribed to by the project. This could have provided useful cost data during the transition, as the States and the local governments move towards supporting local service delivery through different public and private vehicles in the future.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

This project is rated Unsatisfactory due to the modest ratings assigned to the project design, efficacy and efficiency of this project. The objective of increasing agricultural productivity was and remains substantially relevant in South Sudan, although the project objective could have been more relevant by focusing on priority needs (staple crop production) and by including capacity building as a main aim during the transition. The two phase approach adopted by the project’s design put critical capacity building activities at risk during the transition period and failed to put in place a monitoring and results reporting system that could measure the project objective. In a very fragile post-conflict environment, the project achieved some notable outputs, including supporting the organization of farmers groups and the transfer of agricultural technologies and more modern practices, however the information provided on productivity in the project areas – the objective of the project – was not attributable to the project. There is no data provided by the project on tree crop production or the productivity of the forest groups it was supporting. Efficiency is rated modest due to a lack of an economic analysis, cost-benefit, or value for money analysis at appraisal and completion.

a. Outcome Rating: Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

Risk to Development Outcome is rated High. The Republic of South Sudan is a fragile, conflict affected state where risks associated with uncertainties in the security environment highly affect development gains. Conflict has reignited in three of the target states –Unity, Upper Nile and Jonglei. The sustainability of the advisory services system supported by the project was uncertain at the time of project close since the project financed 100% of the Farmer and Forest Group costs. With project close, most of the community facilitators have left the project area, hence the bulk of the farmers are left without advisory services. While some groups are trying to make additional investments and to register as cooperatives, very few groups that did not exist before the project are staying together. Meanwhile, many of the constraints to achieving sustained agricultural productivity are rife – including poor road networks that lack connection to the Southern Greenbelt- South Sudan’s “bread-basket,” climatic conditions including drought, and risk of continued conflict. Key pieces of legislation that were supported by the project were yet to be enacted by project close, and with very little capacity built to date, capacity risks are high. The number of staff trained under project activities is too few to sustainably impact the functioning of the targeted institutions, and the risk of staff turnover is demonstrably high.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
Quality at Entry is rated Moderately Unsatisfactory. Quality at Entry was challenged by a lack of awareness about South Sudan’s political misgivings concerning international donor assistance immediately after re-engagement, unrealistic expectations about implementation capacity and the operating environment, and poor sequencing that deprioritized the institutional and human capacity building that would be needed to revive the agricultural and forestry sectors. Although unclear from the project documents, it appears that the project reinforced some farmer and forest cooperatives that had been previously formed, while forming other groups for the first time. It then invested in an advisory service system that – while during conflict – provided critical support and services. However, after the cessation of major warring activities, the Government of South Sudan was keen to take over that role. As a result, by project close, the apparatus employed by the project is reportedly no longer in place, the community facilitators have left, and many of the farmer groups have disbanded due to lack of financial resources and access to technical assistance. The project lacked a robust monitoring system –which could have been configured as a perception-based survey in the absence of having access to agricultural and forestry data in the project areas. Even so, the system that was developed does not seem to be disaggregated to understand the project’s differentiated effects across the different agro-ecological zones. The agro-ecological zones of the five states targeted - Upper Nile, Unity, Jonglei, East and Central Equatoria -- differ widely. East and Central Equatoria include areas of the Green belt – South Sudan’s most fertile farming lands – whereas the Upper Nile, Unity and Jonglei are dominated by flood plains and their populations are highly dependent on livestock. Since inter-communal conflicts– often based on disputes over land and livestock- is common in these states, the project should have included a description of how it would interact with farming groups in the north – and how the decision to support some groups over others – could potentially affect existing relations and group dynamics. One of the outcomes of this project was extensification of farming systems. This would have affected land and water use decisions and would have required communal negotiation over access and usufruct rights.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
Project Supervision is rated Moderately Satisfactory owing to several course corrections that were made in the project, in despite of the project’s unsatisfactory outcome. The Bank task team was situated in Juba, where it provided prompt support to the implementing teams, particularly with regard to Bank procedures and operations that required consistent oversight and management to ensure delivery of many of the project outputs. During the early supervision period, inadequate support was given to sensitizing the Government of South Sudan to the Bank’s procurement requirements. This weakness is also noted in the IA performance section below. At mid-term, the supervision team introduced a project implementation unit that was better equipped to manage Bank procedures, such as procurement. However the decision to informally restructure the project in March 2011 seems less than cost-efficient and somewhat misguided since the objective was not substantially changed and no new populations were targeted. Although the responsibility of the Government, the supervision team did not provide adequate oversight, attention or resources to support the environmental and social safeguards. And, no efforts were taken to introduce innovative data collection to bolster the ICR, in recognition of the weak M&E system.

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Government performance was hampered by its lack of contribution to the project, as committed at appraisal. It also did not comply with some of the project’s legal covenants, including the preparation of an Environmental Safeguards Management Framework, and a condition of effectiveness ( the hiring of experts from the International Crops Research Institute for the Semi-Arid Tropics) although this condition was later dropped by the Bank. The long delay in resolving this TA issue however slowed down project implementation. Regulatory frameworks for Extension, Forestry and Research that were supported by the project were yet to be enacted at project close. Management Information and Monitoring & Evaluation systems related to these regulations were also not developed by project close.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
The bifurcation of the project resulted, in effect, in two main implementing agents. The Lead NGO as responsible for the delivery of the lion’s share of the project activities, since only phase I was completed. The Lead Agency demonstrated competency in the area of service delivery, as evidenced by the several outputs reported by the project. However there were not surveys or other instruments implemented to assess the quality of the agency’s service delivery function. The Lead NGO appears to have enabled a high degree of gender participation in the project. Here again, there is no assessment of quality however so it is difficult to make a determination as to whether this engagement was meaningful or effective. According to the ICR, the agency was expected to measure crop yields, but the data was spotty and not managed well, further weakening the already frail M&E system. The Ministry of Agriculture and Forestry (MAF) was the designated Implementation Agency. Disagreement about procurement procedures slowed down implementation and the IA did not put in place the required Environmental and Social Safeguards management Framework. Procurement, ESMF, and M&E bottlenecks were partially due to the IA’s dislike of the use of international technical assistance owing to salary differences and a desire to take on these roles locally.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
As indicated in the Final Project Paper (the “PAD” equivalent for MDTF activities), the project intended to design a project specific M&E system, however this was not accomplished. A decision was taken by the Directorate of Planning and Programming - the directorate who would have managed the M&E system – to monitor the project in the context of a sector-wide M&E system. While understood, given the vital data collection needs in the country related to food security, the M&E system did not disaggregate for project level interventions. Baseline indicators referred to in the ICR refer to this baseline sector wide survey undertaken in 2010. It is not clear why more efforts were not made to work with the Lead NGO delivering services, who could have provided project specific information for the mainstay of the project interventions. Data collection, reporting and data management could have been a condition of the partnership. Neither the project nor the information provided on this sector survey includes livestock or forest indicators.

The key project indicators were not monitorable since the M&E system was not constructed in a way that could disaggregate the effects on the project beneficiaries. They were also linked to the phases that have been discussed and explained in this review as being untenable. They were also overly ambitious. There were four main project indicators: (1) Increase in productivity and marketed output achieved by producers participating in the project; (2) Increase in labour productivity (3) Increase in food grain production (phase 2); (4) Increase in proportion of farmers diversifying to high valued crops. In addition to the issues mentioned with regard to the need to disaggregate results, there were no mechanisms established to measure marketing or labor. Other key indicators, like diversification to high value crops, run far ahead of the project and country capacity to implement during the project cycle.

b. M&E Implementation:
The Lead NGO, which was the executing agency that worked closely with farmers, focused mainly on monitoring project inputs. Some crop yield data – including area cultivated were was collected - but these were not systematically sorted, analyzed or compiled in a usable manner. An M&E consultant was recruited almost two years after project effectiveness but remained in the position for only one year of the project cycle. After this departure, the position was not replaced. By the end of the project, the ICR indicates that the project began to assist the Directorate of Planning and Programming to begin to build M&E capacity within State agencies, however this did not have a direct effect on the project reporting.

a. M&E Utilization:
The weak M&E constrained proper reporting on project progress. The information needed for assessing project’s impact, on economic, financial, social and environmental were not available. Indicators were streamlined as a result of informal restructuring, and the overall project objective dropped its quantifiable targets. While there is evidence that there was less uptake than anticipated in planted forest activities, the project does not seemed to have had a mechanism to course-correct and learn from these mid-term findings. The informal restructuring of the project retained the focus on forestry, while removing quantifiable group targets. There is no evidence that a M&E system was used in this project to enhance effectiveness in real –time.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The environmental and social safeguards were not implemented as originally planned. The Government, did not prepare an Environmental and Social Management Framework (ESMF), although, according to the Grant Agreement, this was to be prepared 6 months after project effectiveness. The project Safeguard Specialist attempted to support the development of a mitigation system, however this staff member was only brought in the last year of the project, and to no avail. The required pest management plan was not finalized but as stated by the ICR (p. 9), the use of pesticides by participating farmers was negligible. In addition, several issues related to the expansion of agricultural land should have been picked up and overseen by the Bank safeguard specialist, given the fact that land is at the heart of grievances between groups in South Sudan. The ICR did not include any safeguard compliance rating from the ISR documents or the MTR. The project team subsequently stated that the safeguard compliance ratings were unsatisfactory in the ICR.

b. Fiduciary Compliance:
Financial Management: Financial management of the project was implemented by a Project Financial Management Unit in the Ministry of Finance and Economic Planning (MOFEP). Following the recruitment of a Financial Specialist for the PIU, the overall financial management proceeded smoothly. Financial management of the matching grant by the LNGO faced some problems during the initial two years of the project, because of difficulties in obtaining acceptable receipts from the Farmers Groups, resulting in delayed processing of the Statement of Expenditures (SOEs). The issue was resolved after the LNGO established a proper payment system. While the ICR did not indicate whether the final audit of the project was unqualified, this review utilizes the following statement made available through the last audit report made available that reads: the Financial Statements give a true and fair view of the revenues collected and expenses paid under the project during the thirteen months ending January 31, 2013 in accordance with the International Public Sector Accounting Standards..”


Procurement: The procurement of goods and services was initially carried out from the center by the MOFEP, which was supported by Procurement Agents. However, this did not work as expected, because of weaknesses of the procurement agents. Project procurement was transferred to MAF after the recruitment of a specialist within the PIU. The new arrangement was successful in dealing with the remaining procurement needs of the project, as well as in transferring knowledge to the staff. The only problem which remained throughout project implementation was the poor specification of goods and services by the technical departments, which sometimes resulted in the procurement of goods that were at variance with requirements. This challenge is somewhat endemic in post-conflict, conflict affected and fragile countries.

c. Unintended Impacts (positive or negative):

d. Other:
Gender: As appraised, the project specifically designated women groups as important project beneficiaries, and this was incorporated within the beneficiary selection criteria. During implementation, the participation of women was actively encouraged and the ICR reports that about 48% of farmers in the farmer groups supported were women. This is a very high ratio for World Bank projects and deserves further inquiry. The ICR did not report on the type of female participation, representation, decision-making authority, or the benefits that were directed to women.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Unsatisfactory
While recognizing the substantially relevant aim of increasing agricultural productivity in South Sudan, the

outcome rating is downgraded due to modest design, modest achievement of objectives and modest efficiency.  

Risk to Development Outcome:
Significant
High
Conflict has reignited in three of the target states; the advisory services supported by the project have been disbanded and many of the groups supported lack financial resources to support continued costs. The lack of emphasis on capacity building in this project creates a high risk to sustaining these services.  
Bank Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Borrower Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

Lessons derived from the South Sudan are applicable to many of the Fragile and Conflict Affected States supported by the World Bank Group. Lessons are derived from learning that has occurred since project close- as reflected in the ICR, and World Development Report 2011 on Conflict and Security.

In post-conflict environments, international development assistance should help to build capacity for medium-to-long term service delivery functions while tending to imminent development needs. NGOs can play a supportive role in service delivery, but during transition, priority should be placed on enabling local governments to gradually take responsibility for development programs in their jurisdiction. In South Sudan, local governments have depended on NGOs for their operating expenses. Very little attention was paid to the role of Local Governments during the transition, with regard to their assumption of responsibility or with regard to their role of providing local oversight of private service delivery functions.

Development assistance in fragile environments should ensure that expertise and capacity remains in program areas to continue assisting beneficiary communities. Facilitation teams trained by projects to provide technical assistance, whether in the agriculture, forestry or other development sectors, should be institutionalized to allow a continuation of these services after project close. (South Sudan ICR).

In a post conflict environment the ability to be flexible with regard to choice of service providers is a factor for project success. In South Sudan, ex-ante selection of institutions to provide certain services/inputs, and conditioning this to project effectiveness created delays in project implementation.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The rating of the Quality of ICR is ‘satisfactory’ but only marginally so owing to a lack of data collection at end-line. The ICR was comprehensive with a candid report of implementation challenges and adequate formulation of lessons. The ratings were realistic. However, the ICR could have been more clear about the lack of evidence with regard to the achievement of the project’s objective. The Efficacy section was unclear with regard to the instruments that were used to measure overall progress in the agricultural sector versus project attributable information. A summary of this information and the methods used to collect and report data could have been provided in the annexes. More could have been done during the ICR mission to supplement for this lack of data, by implementing a perceptions survey about the project, to the targeted beneficiaries. This assessment could have provided useful information about attitudes concerning the adoption of new technologies, the performance of the lead NGO, etc. The ICR indicates that an ERR was not calculable owing to a lack of data, but some basic cost-benefit analysis could have been run by the ICR team. Safeguard compliance ratings, audit outcomes and financial management supervision and procurement ratings were not presented.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions