Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Sd - Livestock And Fisheries Development Project - Mdtf


  
1. Project Data:   
ICR Review Date Posted:
11/18/2013   
Country:
Sudan
PROJ ID:
P101912
Appraisal
Actual
Project Name:
Sd - Livestock And Fisheries Development Project - Mdtf
Project Costs(US $M)
 42.0  16.57
L/C Number:
Loan/Credit (US $M)
 20.0  13.49
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  11/07/2006
 
 
Closing Date
12/31/2008 06/30/2011
Sector(s):
Agricultural extension and research (60%), General agriculture fishing and forestry sector (40%)
Theme(s):
Rural policies and institutions (67% - P) Rural services and infrastructure (33% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Lauren Kelly
Ridley Nelson Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:

Original Project Objective: To improve the performance of the Livestock and Fisheries Sectors in five selected States of Southern Sudan through capacity building, improving animal health, reducing post harvest losses and improving marketing infrastructure (Grant Agreement February 27, 2007 Final).

Revised Project Development Objective: To build capacity of the public sector in Southern Sudan to fulfill its mandated role in livestock and fisheries and to promote continuity in provision of key services in the livestock sector in five selected states of Southern Sudan (Third Amendment to the MDTF Grant Agreement, November 18, 2010).

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 11/18/2010

c. Components:

(1) Institutional Development and Capacity Building (Appraised value was US$11.25m; actual value was $7.04m). Support for capacity building of the Ministry of Animal Resources and Fisheries (MARF) to formulate policies, design appropriate legal instruments and set priorities, build technical and managerial skills at senior levels of MARF and the five State Directorate of Animal Resources and Fisheries, and needed infrastructure (construction of MARF headquarter offices and rehabilitation of five State Directorates), equipment, and vehicles. The component was also designed to finance funds for covering recurrent costs and conducting studies that would support sectoral policy and institutional development.

(2) Improvement of Service Delivery for Animal Health (Appraised value was US$17.00m; Actual value was US$8.32m). Improve livestock production and marketability through disease control. This was to be achieved by financing the services of NGOs to support local animal health services; the construction of one central laboratory in Juba and two regional laboratories in Wau and Malakal; expansion of the Marial Lou Livestock Training Center; provision of regional experts to impart training to trainers in livestock related topics; financing of contingency drugs and cold chains, cost of mobile clinics and requisite laboratory, as well as field diagnostic equipment and supplies.

(3) Livestock Marketing Support (Appraised Value was US$8.40m; Actual value was US$1.21m for both Component 3 and 4). Support the development of market facilities and peri-urban dairy, and comprised: the provision of funds for price-monitoring and dissemination; staff training; promoting services at the market points; conducting studies on the feasibility of physical facilities i.e. slaughter houses, cattle holding and auction yards and water supply services, and the construction of these if warranted by the studies; promotion of peri-urban dairy; and the financing of a suitable organization to implement the livestock marketing support component.

4) Development of Fish Production and Marketing (Appraised Value was US$5.35m; Actual value was US$1.21m for both Component 3 and 4). Support the development of an effective service delivery system for the development of fisheries resources, minimizing post-harvest losses, improving access to markets ;and building capacity of field officials. Main activities included: financing the expansion of the training facility at Padak; provision of training to trainers and other staff; training of fishers and demonstration of better fishing techniques; and developing extension services. Following, technical and financial assessment of the market potential for fish and the interest of the communities, the project would improve fish landing and other facilities with community participation during the second phase of the project.

Following restructuring, components 3 and 4 were merged into one, under a new heading: Support to Livestock and Fish Marketing. Therefore, as restructured, the project had three components: i) institutional development and capacity building; ii) improvement of service delivery for animal health; and iii) support to livestock and fish marketing. The purpose of the latter was to identify entry points for improving the marketing of livestock and fish products. The last component also aimed at supporting livestock value chain studies and piloting a peri-urban poultry development initiative.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Costs and Financing: The actual cost of the project was US$16.57m, or 39 percent of appraised value, due to the Borrower’s failure to fulfill its commitment to the project and the subsequent decision to reduce its scope.

Borrower Contribution: The Borrower did not fulfill its appraisal commitment of US$22.0m, contributing only US$3.08m, or 14% of its original commitment. While the multi-donor trust funds were fully disbursed by project close, the first half of the project period was characterized by serious disbursement lags: by November 2008, only US$1.2m had been disbursed.

Dates: The closing date was extended by two-and-a-half years from December 31, 2008 to June 30, 2011 to permit completion of project activities. It should be noted that this project was implemented under the transitional government, before South Sudan became a state on July 9, 2011.

Restructuring: The project revised its objectives and outcome indicators in November 2011. seven months prior to the project's extended closing date. By the time that the project was approved, 78 percent of the project funds had been disbursed.

Note on the Evaluation of Restructured Projects: If a project’s development objectives have been formally revised (through a restructuring approved by the Board and resulting in an amended legal agreement), the ICR and the ICR Review records both the original and formally revised objectives. Project outcome are then assessed against both the original and revised project objectives.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
The Relevance of the Original Objective of enhancing the performance of the livestock and fisheries sectors is rated Modest. In a country ravaged by over fifty years of war and therefore challenged by an extremely low level of physical, human and institutional development, the objective of improving sector performance was overly ambitious, reflecting an unrealistic appraisal of government capacity at the time. The first country strategy for South Sudan, the ISN FY13-14, builds on the lessons of the Bank’s own analysis of the projects financed by the Multi-Donor Trust Fund for South Sudan (MDTF-SS). This analysis corroborates the conclusions of several other studies conducted for the MDTF-SS that found that activities had been challenged by excessive expectations raised by the Comprehensive Peace Agreement (CPA), donors’ unrealistic expectations of government capacity, and an underestimation of the challenges that would be faced in implementation. In its 2011 Brief, World Bank Group Engagement in South Sudan: Lessons from Evaluation,IEG pointed to the need to adjust projects to the South Sudanese operating environment, with designs characterized by a few well defined objectives, realistic implementation schedules, and simple procedures. The focus on economic recovery, especially for the large segment of the population that depend on livestock and fisheries, should be recognized as being a relevant project aim, and is thus reflected in the modest, versus negligible, rating.
The Revised Objective, focusing on building public sector capacity to promote continuity in provision of key services, is rated Highly relevant. The relevance of the revised objective is in line with the Interim Strategy Note (ISN) FY13-14, that emphasizes the need to help create credible national institutions in the long-term while also delivering short-term tangible benefits. This objective was realistic. The focus on building capacity for effective service delivery to the large segment of the population that depend on livestock remains relevant, especially after independence, as NGO support for basic service delivery winds down, and responsibilities for delivering animal health services are transferred to the public sector.

b. Relevance of Design:

The relevance of the design to the original objectives is rated Negligible. This rating is due to a significant disconnect between the project objectives, the phasing of the project, and the project funding -- which was less than what was needed to achieve the project objective, The project was designed in two phases. The first phase was to be implemented over the first two years. This phase would focus on capacity building and service delivery. The second phase, to be implemented over the remaining three years, would concentrate on longer term institutional development, while continuing with service delivery and marketing activities. According to the ICR (page 5), it was known during project preparation that a firm commitment by the multi-donor group had been made to the first phase only. The grant agreement also specified that the first phase would close after two years following effectiveness, but project design maintained the structural link between the two phases, including the four part objective and a results framework that covers both phases.

The relevance of the design of the restructured project is rated Substantial. Scaled back and realistically designed to achieve what was afforded by project means, the revised objective was accompanied by a results framework that accurately reflected project activities. Activities planned for the second phase, such as livestock marketing, market information systems, fish production and marketing were dropped since funding was not available under a second phase. The new results framework was more directly aligned with the project objective. The new framework introduced new outcome indicators such as "continuity in animal health service delivery" and retained input and output indicators that could help establish the link to the revised projected outcomes, such as "number of community animal health workers and veterinary pharmacists trained" and "number of households receiving animal health services reached."


4. Achievement of Objectives (Efficacy) :


The original objective of achieving improved performance of the Livestock and Fisheries Sectors in five selected States of Southern Sudan through capacity building, improving animal health, reducing post harvest losses and improving marketing infrastructure was Modestly met. Evidence is provided for only one intervention area, related to the improvement of animal health. There is evidence that skills were transferred from the international NGO community - which had been delivering vaccination services during the war years - to a cadre of community animal health workers that resulted in widescale vaccination of 580,000 livestock - of which some 380,000 animals were vaccinated against specific disease threats. The evidence allows this review to reasonably conclude that the vaccination of the livestock would have contributed to enhanced livestock sector performance in comparison to the counterfactual of not vaccinating the livestock. There is little further evidence of improved sector performance or outcomes. Performance was not improved in areas such as reducing post harvest loss and improving marketing infrastructure. Project components that were originally designed to achieve the construction of marketing infrastructure, development of market information systems, demarcation and provision of watering points along livestock trading routes and the development of fish production and marketing were dropped as the activities had been planned for a second phase for which funding was not available. There is no evidence that the fisheries sector performance was improved. The only information made available in the ICR with regard to the Fisheries sector is the confirmation that two fishing shops were constructed, and that these shops are intended to serve two cooperatives of about 500 members that require access to fishing gear. Progress on supporting the legal and regulatory framework for both the livestock and fisheries was stalled. By the time of project closure, the GoSS had undertaken work in this area, but the work was still in draft. A review of the project supervision materials indicates that USAID and EU funded projects may have assumed responsibility for these project aims.

The revised objective of building capacity of the public sector in Southern Sudan to fulfill its mandated role in livestock and fisheries and to promote continuity in provision of key services in the livestock sector in five selected states of Southern Sudan was Substantially met. The revised objective places an emphasis on building sector capacity and ensuring the continuation of vital services. For a country emerging from a devastating 22 year civil war with massive destruction of the country's infrastructure and institutions that were just being reformed under a transitional government, the objective of building public sector capacity should be viewed from a very basic lens. For example, the project supported the reconstruction and construction of essential sector infrastructure. The project supported the physical construction and/or renovation of five State Directorate offices which, according to the ICR, had the effect of improving working conditions for sector staff. Three States were provided with new office blocks and two States had their old offices refurbished. Offices were equipped with generators, computers, furniture and equipment. The project also supported the rehabilitation of a regional veterinary laboratory at Wau, dormitories and classes at Marial Lou Livestock Training Center. All of these outputs were essential for providing basic working conditions for sector staff to gradually progress to the stage of impacting sector performance. Other outputs that helped to enable the public sector to perform their sector duties included the procurement and distribution of 28 vehicles and 43 motorcycles to the states.

The project was also effective in building capacity to promote the continuity of key services. As described above, there is evidence that the project supported capacity development of a National Community Animal Health Service and that that capacity has had the effect of providing key animal health services. Thirteen hundred national Community Animal Health Workers were trained and equipped to perform early detection, disease reporting, treatment and vaccination. This transfer of skills and knowledge enabled the public sector to deliver vaccinations to 580,000 livestock, 380,000 of which were treated against identified disease. The without project scenario may have resulted in the vaccinations taking place anyway, but the capacity would have remained with the international NGO community, threatening the sustainability of these services over time. Other outputs that helped to enable the public sector deliver key services included the construction of twenty veterinary pharmacy/drug stores at the county level and the procurement of twelve mobile veterinary clinics (although a recent visit by the ICR Review author to Juba revealed that not all of the vehicles and their diagnostic and treatment equipment are being used for their intended purposes).

One of the weaknesses of the project's support for capacity development was the low level of attendance of state level staff. Seventy-five percent of the trainees are based within the central government. As discussed in the section on M&E, another weakness pertains to the absence of any criteria for measuring learning or uptake and application of the offered training. Some 320 staff were trained as a result of the project in livestock related services.

5. Efficiency:


Efficiency is rated modest overall. No qualitative or quantitative assessment of anticipated project benefits was made to explain the project’s net benefits at the design stage. The efficiency of the project was hampered by implementation delays. These included: (1) making government contribution of funds a condition of effectiveness - a condition that was ultimately dropped but that caused an initial six month delay; (2) the failure of the Government of the GoSS to contribute its committed share -- creating an environment of anticipation around funding shortfalls that detracted from implementation; (3) the freezing of recruitment of international technical assistance - a key feature in project design not favored by the GoSS; (4) the lack of procurement expertise and an overestimation that expertise could be fully integrated into MARF by the second year of the project.

The ICR offers a back-of-the envelope cost benefit assessment associated with the reinstatement of the public animal health service delivery system. According to the government, the project’s main contribution has been the restoration of a functional animal disease surveillance and control mechanism and healthcare system. The total cost of the animal health component was about US$ 5.05 million, which amounts to about US$0.66 per animal unit. This should be regarded a cost effective undertaking given the important results achieved in animal health services. The project also vaccinated and treated about 580,266 and 389,614 animals, respectively, and this has benefitted livestock owners by reducing morbidity and mortality. Practitioners estimate that the vaccinations provided would have reduced mortality by about 5% and that the project’s treatment activities would have reduced mortality by at least 10%. Based on these assumptions, project benefits from vaccination and treatment alone would be about US$16.22m.

Other achievements - in relation to capacity building - are difficult to quantify. According to the GoSS, the improvement in working conditions and the provision of equipment and other facilities have significantly raised the operational efficiency in MARF and the States. Some staff indicated that, with the project, productivity in the Ministry has increased several fold.


a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The overall project rating is Moderately Unsatisfactory. For projects whose project objectives have been formally revised the project outcome is assessed against both the original and revised project objectives. The separate outcomes have been assigned ratings and have been weighted in proportion to the share of actual loan/credit disbursements made in the periods before and after approval of the revision. In the case of the South Sudan Livestock and Fisheries Development Project, the disbursement ratio is 78% prior to the revision; 22% after the revision approval. The original project outcome is rated Unsatisfactory, based on a modest relevance of the original objective and a negligible rating for its design, modest efficiency and modest efficacy. The revised project outcome is rated Moderately Satisfactory, based on High Relevance of the revised objective and a Substantial Relevance of design ( the project dropped many of the components that had not been feasible under the original design), Substantial Efficacy and Modest Efficiency. Applying the weighted formula based on the project disbursement ratio,

(.78) x (a numerical value of 2 for an MU rating) + (.22) x (a numerical value of 4 for a MS project) the project outcome score is 2.44. Mathematically this would be Unsatisfactory, fractionally (0.01) below Moderately Unsatisfactory.

Given the fact that the borrower's qualitative assessment was conducted in a country situation where quantitative assessment would have been very difficult to determine whether public sector capacity was significantly enhanced - and given that there is some ambiguity about the extent to which the original objective of improved "performance" meant sector outcome performance alone or included improved public sector services performance - IEG finds the rating of outcome to be Moderately Unsatisfactory (rather than UnsatisfactorY). This rating has also been assigned in consideration of the counterfactual, with regard to the potential effects of not vaccinating the 580,000 livestock, both from the point of view of human survival and the potential for disease spread within both the country and the region..

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

The ICR notes that there is a high risk to the development outcome of the animal health service system but only a low risk to capacity development, and therefore rates Risk to Development Outcome as Substantial (Significant). Not enough information is provided in the ICR to assess risks to development outcome otherwise. There is a reference to a risk that MARF may fail to provide sufficient funds to purchase drugs and vaccines - but more information is needed about the current status of the 20 pharmacies and how their design as business ventures is fairing. There is also a reference in project documentation to some of the laboratory vehicles being stripped of their equipment and used for other purposes. There are risks exogenous to the project that were not addressed in the ICR, like severe droughts, and this could pose an institutional risk if contingencies are not in place for the livestock sector. There are also political risks in relation to the negotiations with Sudan and economic risks associated with the price of oil - that will affect the GoSS budget and its ability to support public investments in the livestock and fisheries sector. There is no reference in the ICR to the wage bill or the salaries of the community animal health workers.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:

The quality at entry is rated Unsatisfactory. The project as originally designed was insufficiently flexible to allow for changes in response to actual circumstances. The Bank at entry misread the environment in which it would be working. GoSS Ministerial staff had been displaced during the 22 year conflict. Many of them went abroad and worked for and with other African governments. These staff brought with them a mistrust of the World Bank and foreign aid, having lived through structural adjustment, and also skepticism about the relationship that the World Bank had with its northern neighbor during the conflict. There was a misreading by the Bank of how the transitional government would view international technical assistance. In the event, they questioned why a substantial amount of the project finance was flowing into international technical assistance to support what was perceived to be heavy administrative procedures (procurement, financial management etc).
Implementation arrangements were overly complex and responsibility was placed within a directorate that lacked the authority or ability to coordinate across several technical directorates, five states, and with key Ministries. As discussed in Relevance of Design, project preparation (including the delineation of the project objectives and the development of the results framework) proceeded with a two phased approach although it was unclear that financing for the second phase would later be available.

Quality-at-Entry Rating: Unsatisfactory

b. Quality of supervision:
The Quality of Supervision is rated Unsatisfactory. The fist half of the project period was characterized by implementation delays and unanticipated funding shortfalls. According to the Borrower's Summary (page 28), the first two years were virtually lost. The main obstacles were procurement and financial management, which were eased when the World Bank and the borrower agreed to recruit expert technical assistance in these areas at mid-term (April 2009). Changes introduced to project management, including the establishment of a Project Coordination Unit under the Director General for Special Projects enhanced the effectiveness and speed of project implementation. However, based on the final results of this program, it is not clear why at mid-term (April 2009) the original project objectives were considered relevant and why the two phases were maintained -- especially since the implementation supervision mission conducted the following year (April 2010) recommended a fundamental restructuring of the project and its expected results. The ICR notes that there were few comprehensive review missions that provided a complete picture of project implementation. Supervision towards the end of the project cycle reflected a realization about the lack of realism of the project objectives and the weakness of its design. The revised objective and outcome indicators were more relevant for the South Sudan context at the time, however project restructuring occurred too late in the project cycle to significantly affect its outcome and the pre-restructuring performance is dominant in this rating..

Quality of Supervision Rating: Unsatisfactory

Overall Bank Performance Rating: Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Government performance is rated unsatisfactory for a number of reasons. The Borrower did not fulfill its appraisal commitment of US$22.0m, contributing only US$3.08m, or 14% of its original commitment. Implementation was initially delayed by six months because the Government had not met the conditions for effectiveness, which were subsequently waived by the Bank. The Government then froze the hiring of technical assistance on cost grounds, which stalled important activities. The Government’s failure to recruit an Environmental Specialist for the project meant that the required safeguard instruments were not prepared. Finally, the ICR notes that the Government tolerated that “project procured vehicles in the States be taken away by senior officials of the State governments to be used in other departments, which reduced the mobility of the technical staff in providing improved animal health services.”

Government Performance Rating: Unsatisfactory

b. Implementing Agency Performance:
The performance of the implementing agency is rated moderately satisfactory, since several of the delays that are associated with this project were a product of faulty project design rather than implementation challenges. The issues pertaining to financial management and procurement are also mainly attributable to the set up of the multi-donor trust fund, expectations about the capacity of the nascent institutions and their staff, and a lack of awareness about the attitudes and perceptions of members of the implementing agencies about the use of international technical assistance as well as the Bank's procurement and other fiduciary procedures. The implementing agency was managing several projects financed by the MDTF and the EU and was doing so within a year after emerging from 22 years of conflict. Both the Undersecretary and the Special Projects coordinator actively monitored and reported on the physical works, including spending over forty days in the field outside of Juba, meeting with implementing partners and project beneficiaries. There were significant coordination problems in the project, which is equally attributable to the overly complex institutional arrangement that was established by project design. By project end, the MDTF funds were fully disbursed and the Phase I service delivery activities pertaining to the livestock sector were effectively carried out.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
M&E design is rated negligible. Recognizing that data would have been scarce in a country recovering from a 22 year conflict, and that the multi-donor trust fund projects were the first series of development interventions to be implemented following years of humanitarian assistance - the latter of which require different data sets to determine needs and outreach - a more practical and measurable results framework could have been established by the project during preparation. The initial results framework was hinged on the development of baseline data, that was not completed until seven months before project close. Rather, an institutional assessment during the first year could have established measurable areas in which capacity development could have been assessed. Likewise, an assessment of the capacity of the animal health workers and affiliated service delivers need not have been comprehensive but rather representative of the types of skills and equipment that were lacking and the type of skills and provisions that the project planned on imparting to this beneficiary group. In a post-conflict setting, support for service delivery is partly about the services being delivered and partly about restoring legitimacy and trust to maintain the fragile peace. A simple perception survey could have been administered to a random sample of community health workers and the users of their services - covering all five states - to measure attitudes and perceptions as well as the quality of the training and the services being delivered.

The preliminary results framework (Annex 6 of the Project Proposal) was designed so that data collected by a baseline survey could be plugged in. It was data - rather than perception- heavy. most notably, it lacked an indicator of capacity development -one of the core functions of the project.

b. M&E Implementation:
Rather than a single project M&E results framework, the Ministry of Agriculture and Forestry worked together with the Ministry of Animal Resources and Fisheries to produce an overall M&E framework for the two multi-donor projects that were being implemented at the same time - the Livestock and Fisheries Development Project and the Agriculture and Forestry Development Project. The M&E framework and an assessment of project progress was undertaken by an independent consultant, utilized data collected by the Bank supervision missions, and delivered in May 2010. Prior to restructuring, the external M&E work and review of the progress of the Bank's projects included recommendations on how to strengthen the project indicators. These are noteworthy and reflect an understanding of the ways in which the project was seeking to effect behavioral change - in addition to support for direct service delivery outcomes and improved working conditions. These indicators included measuring capacity (technical skills, capacity for inquiry and analysis, for undertaking operations) and the quality enhancement of information and reporting alongside the timeliness of planning and activity implementation. They also included indicators to measure the adoption and utilization of the new extension methods versus the way that services were being delivered by the NGOs - what are the perceptions and attitudes of the user? While inserting these indicators at the time of restructuring would have not been appropriate - the project had disbursed all of its funds and may not have time to design and implement the type of monitoring and evaluation that would have been needed to answer these queries - it is this kind of thinking that should have been reflected in the original design of project M&E.

a. M&E Utilization:
The M&E system established under the project was not utilized due to its delayed establishment and to the fact that the staff in the States lacked the physical equipment necessary to facilitate data collection and transfer.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The Project was classified as Environmental Category "B," and OP 4.01 "Environmental Assessment" was the only safeguard triggered. Nonetheless, the project was expected to use three MDTF-wide instruments, an Environment and Social Management Framework, a Resettlement Policy Framework, and a Medical and Veterinary Waste Management Plan. These had not been developed by project close. Towards the end of the project, an Environmental Specialist was recruited to conduct an environmental audit. As noted in the ICR, "the findings of this assessment indicated some inadequacies in disposing of medical waste and expired drugs, as these were being burnt and buried, instead of being properly incinerated. Likewise, concerns were expressed with regard to the poor protection of staff who handle veterinary drugs and vaccines. [...] The environmental audit report includes suggestions for correcting these deficiencies, and MARF is in the process of purchasing incinerators to deal with the drug disposal issue.”

Although OP 4.12: Involuntary Resettlement was not triggered at appraisal the ICR reports that “inadequate compensation was reported with regard to the relocation of seven families from Government land. If resettlement became an issue during project implementation and the MDTF-wide resettlement policy framework had not been developed, a resettlement policy framework specific to this project should have been prepared instead.

b. Fiduciary Compliance:

The project was characterized by weak financial management during the initial years due to capacity limitations within MARF. However, these difficulties were reduced after the recruitment of a Financial Management Specialist in May 2009. Financial reports and financial management statements were thereafter submitted as envisaged and external audits were prepared regularly. Procurement was a major issue between the GoSS and the World Bank that pervaded all projects managed by the Multi-donor trust fund. Although the Government put in place a centralized procurement agency within the Ministry of Finance and Economic Planning with the support of a consulting firm as procurement agent and an individual consultant as Procurement Advisor, procurement did not work as expected due to : (1)inadequacy of hands-on support provided by the Procurement Advisor; (2) the inability to meet the requirement of the procurement plans and bidding document due to the constraints posed by operating in South Sudan at the time; (3) execution of awarded contracts proved difficult because of the imposition of sanctions on Sudan.

Performance of one of the NGOs was poor - and a failed procurement process for vaccines and drugs for animal treatment hindered the process. The flawed procurement process was stopped and managed by both the Implementing Agency and the World Bank.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Risk to Development Outcome:
Significant
High
 
Bank Performance:
Unsatisfactory
Unsatisfactory
 
Borrower Performance:
Unsatisfactory
Moderately Unsatisfactory
The Implementing Agency performed moderately satisfactorily. It had to overcome faulty design and oversaw significant progress in the re-establishment of a national animal health system.  
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Lessons are derived from IEG's work in Fragile and Conflict States (Lessons 1) and from lessons learned and reported in the ICR (Lessons 3 and 4) with some adjustment of the wording.

In fragile and post-conflict settings, development assistance should be flexible, objectives should be realistic, and projects should be designed in a manner that allows them to adapt to changing circumstances. Results frameworks should be aligned with identified projects risks.

In the case of countries emerging from conflict and those with nascent institutions, conditions for implementing projects should be kept to a bare minimum as they will have difficulties in fulfilling them in time risking project failure. In the case of difficulties in meeting some of the conditions by such countries, consideration should be given by the Bank to reviewing them and granting waivers, provided that these do not jeopardize the project’s outcome.

The case for integrating business opportunities and business development in service delivery to enhance sector performance and support sustainability of the system should be considered in the conceptual design stages of service delivery projects even in post conflict and fragile environments. While training and equipping community animal health workers is an important step in creating a timely disease surveillance and treatment system, service providers require access to a sustainable and affordable source of vaccines and treatments. The private supply chain model that was developed under this project could develop into a sustainable model for expanding private veterinary services in the future in South Sudan. It may offer lessons for setting up service delivery models elsewhere.

14. Assessment Recommended?

Yes
Why?
A cluster evaluation of the MDTF activities implemented in South Sudan could provide input to the MDTF Review planned by IEGCC as part of the Fragile and Conflict Affected State Evaluation.

15. Comments on Quality of ICR:

The ICR is concise and clearly written. The ICR contains most of the elements necessary to evaluate the project. The quality of evidence and analysis are adequate. The lessons drawn are appropriate and based on project experience.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions