Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - South Kordofan State Emergency Project


  
1. Project Data:   
ICR Review Date Posted:
12/09/2013   
Country:
Sudan
PROJ ID:
P101388
Appraisal
Actual
Project Name:
South Kordofan State Emergency Project
Project Costs(US $M)
 14.74  13.7
L/C Number:
Loan/Credit (US $M)
 7.88  7.32
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  06/25/2007
 
 
Closing Date
10/29/2010 10/31/2013
Sector(s):
Animal production (35%), General agriculture fishing and forestry sector (35%), Crops (30%)
Theme(s):
Other rural development (34% - P) Other social development (33% - P) Gender (33% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Lauren Kelly
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The project had two development objectives as identified in the Final Project Proposal (the final appraisal document used by the Bank for projects financed under the Multi-Donor Trust Fund for Sudan). These objectives were to: "(1) contribute to the supply of basic services and facilities to the respective conflict affected populations and to (2) contribute to capacity building of the state government [South Kordofan] to enable it to more effectively execute its mandate, especially at the local (decentralized) level." The ICR objective statement is the same as the Final Project Proposal.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

The project has eight components. An additional activity, financed by the World Bank’s Gender Action Plan, was also implemented by the Project Management Unit and is viewed by the project team and the government of South Sudan as a contribution to this project.

1. Water Supply (Appraised and actual value is US$2.47 million). Executed by UNICEF, this component was designed to finance the installation of 20 new hand pumps, 5 new water yards, and to rehabilitate 100 hand pumps. This component included procurement of a rig, new hand pumps, spare parts for old hand pumps, geophysical studies, training and monitoring.

2. Rural Roads Maintenance and Spot Improvements (Appraised value is US$1.8 million; Actual value is US$6.3 million, including the US$4.5m spent by the State Government of South Kordofan from its own budget to implement component 2 of the Project). This component was designed to finance the clearing of 50 km of rural secondary roads, the grading of 8 km of road, the construction of 3 bridges and 8 culverts. During implementation, the government provided an additional US$4.5m for public works, enabling the construction of a total of 115 km of road, 50 bridges, and 17 box culverts.

3. Basic Education (Appraised and actual value is US$2.84 million). Executed by UNICEF, this component was designed to support the provision of desks and chairs for 48,000 children, school kits for 287,000 school children, 350 teachers’ kits, and 3,500 cabinets for 1,255 schools. (Actual number of cabinets distributed was much lower than planned, with 300 cabinets distributed rather than 3,500 cabinets; desks and chairs were provided to 49,600 rather than 48,000 schoolchildren).

4. Vocational Training (Appraised and actual value is US$0.52 million). Executed by the United Nations Industrial Development Organization (UNIDO), the component financed the rehabilitation of the Kadugli Vocational Training Center to support the training of 200 students in various technical vocations. An additional focus on training youth and women was added to this component.

5. Humanitarian Demining (Appraised and actual value is US$1.6 million). The component supported demining activities along 1.5 million meters of high priority suspected mined areas.

6. Rural Households Livelihood Support (Appraisal and actual value is US$0.60 million). Executed by UNIDO, the component aimed to supply tools to 4,000 households, provide TOT training for tool welding, provide animal traction equipment (Nuba Hoes) and train 400 farmers in the use and maintenance of animal traction equipment.

7. Capacity Building of State and Local Government: (Appraised and actual value is US$3.10 million). The component was designed to support the establishment and maintenance of a project management unit and improved management capacity in 6 ministries (Finance, Economy and Investment, Local Government, Water, Education and Public Works). It aimed to support the construction of 20 offices and 10 living quarters for local authorities, as well as the procurement of 20 vehicles and other equipment intended for local authorities. This component was significantly redesigned so that resources were mainly concentrated in the PMU. It supported the hiring of a consultant within the PMU to undertake a study of institutional capacity building and the training of 1,098 civil servants. Vehicles were not distributed as planned owing to the State Government's concern that the vehicles could become high-jack targets for the rebels. Financing intended for the procurement of vehicles was instead directed towards the refurbishing of 25 rather than 10 state dormitories.

8. Anti-malaria Bed Nets: (Appraised and actual value is US$1.0m). This component executed by the PMU was designed to provide long-lasting insecticidal bed nets to cover about 80% of the rural population and training of the bed net users. This component was scaled down owing to the donation of thousands of bed nets from other aid agencies over the course of the project period. Accordingly, the component only procured 8,500 bed nets, which together with total donor contributions, allowed for a reported coverage of 80% of the rural population. Remaining funds were used to support the procurement of other needed equipment, including 25 fogging machines, 2.5 tons of insecticides and an X-Ray machine - to assist the Ministry of Health strengthen its anti-malarial activities.

* An additional activity, or sub-project, was added to the project with the use of a US$200,000 grant from the Bank's Gender Action Plan. The grant supported a Tundia Area Women Development pilot initiative (Tundia is a neglected area in the Nuba mountains which has been was most affected by the civil war). The grant, implemented by the PMU, financed the construction of two women's development centers and helped establish a women’s development committee whose members participated in a study tour to Kassala state.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: Total project costs were appraised at US$14.74 million. According to the ICR, actual costs were US$13.7 million, or 93 percent of the appraised value of the project. However, these do not include US$4.5 million provided by the State Government from its recurrent budget for the roads component, and which enabled the scaling up to 115 kilometers of roads and the building of 50 bridges and 17 box culverts.
Financing: External financing was provided from a Multi-Donor Trust Fund (MDTF). The Trust Fund disbursed US$7.32 million, compared to an appraisal estimate of US$7.88 million.
Borrower Contribution: One of the key features of the MDTF was the unprecedentedly high contribution agreed to by the Government of National Unity: counterpart finance was to equal 50 percent of MDTF project financing. However, this commitment was consistently defaulted on across the MDTF portfolio. As a result, all three emergency projects implemented under the MDTF, including this project, experienced shortfalls in counterpart financing resulting in significant implementation delays and set-backs. In this case, the Government of National Unity agreed to provide US$6.86 million. By project close, the Government had contributed US$2.9 million, or 42 percent of the agreed amount. The funding gap was filled by the State Government of South Kordofan (US$2.2 million) and the Unity Fund of Sudan (US$2.3 million). Together with the Government's contribution, the total Recipient funding at project closure was US$7.4 million (ICR, Annex 1). As noted above, the State Government contributed a further US$4.5 million from its recurrent budget to the implementation of the project's road component.
Dates: At the time of Project approval, the MDTF was not subject to regular Bank procedures, so did not require Board Approval. In-country approval took place on June 25, 2007. Project effectiveness was planned for August 2007, but was extended by six months to February 2008. Envisioned as a one-year program, the original closing date was scheduled for May 31, 2008. This was extended three times between 2008 and 2010, and the project eventually closed on October 31, 2011.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Both objectives are Highly Relevant. Although designed prior to its publication, the project objectives are closely in line with the recommendations of the 2011 World Development Report on Conflict, Security, and Development, including delivering quick visible wins, taking into account stress factors and potential drivers of instability, and the need to focus on capacity building of weak institutions. They are also relevant to Sudan's 2007-2011 Five Year Strategic Plan and the goals laid out in the World Bank-UN Joint Assessment Mission and the Comprehensive Peace Agreement. The 2005 Comprehensive Peace Agreement (CPA) made a provision for 70 percent of National Development Reconstruction Funds to be targeted to the least developed states in Sudan (Abyei, Blue Nile, and South Kordofan), reflecting the recovery and development needs of these areas as well as their pivotal role in sustained peace and security. By March 2006, members of the first Sudan Consortium noted that since the signing of the CPA, these areas had not received the expected resources, a factor which would aggravate political instability in the region. The quick formulation and start-up of a recovery project –including quick impact interventions – was in direct response to this need. The objectives are relevant to the Interim Strategy Note (ISN) for Sudan for the Fiscal Years 2013 and 2014 that focused on increasing resilience by helping to build credible local institutions to establish the authority and capability of the state to deliver services. As noted in the ISN, while the large presence of bilateral donors and international non-governmental organizations (NGOs) has focused on short-term delivery, the Bank's comparative advantage lies in the area of longer-term institutional building. At the same time, the Bank would take advantage of short-term opportunities that arise to make tangible improvements in livelihoods and service delivery.

b. Relevance of Design:
Relevance of Design is rated Substantial. The project has a clear statement of objectives that is linked to measurable intermediate and final outcomes. The causal chain between the activities financed and the aim of contributing to the supply of basic services and facilities to conflict affected populations was clear. The project partnered with UNICEF who had a presence on the ground in the water and education sectors, to support basic service delivery, and UNIDO to supply tools and farm equipment. The causal chain is less clear with respect to building state and local capacity. The activities financed by the project supported physical and technical inputs and outputs necessary to begin to build state and local capacity, but the results framework lacked a mechanism to track the outcomes of the capacity development activities. Sequencing was an issue in this regard, since one of the project components was designed to undertake the baseline assessment of institutional capacity building.


4. Achievement of Objectives (Efficacy) :

(i) Contributing to the supply of basic services and facilities to the respective conflict affected populations. High. The project delivered basic services to some 914,794 conflict affected persons, exceeding the project target of 900,000 persons. An additional 275,000 conflict affected persons were estimated to have benefitted from road improvements. Specifically, the project:
  • Surpassed its target for rural road rehabilitation: 115 kilometers of secondary roads were improved compared to the target set for 50 kilometers (a 230 percent achievement).
  • Verified or cleared mined areas and roads (or those suspected of being mined) in the 40 communities targeted at appraisal.
  • Increased access to improved water sources for all 90,000 persons targeted, and although not originally targeted, 50,000 livestock also benefitted from the improved water sources.
  • Delivered all 287,000 school kits as planned and rehabilitated the Vocational Training Center at Kadugli.
  • Distributed 4000 hand tool kits - in addition to 400 animal traction tools - were distributed as planned.
The project fell short of its original target of delivering 37,500 long-lasting insecticidal bed nets. However, this target was revised downward at the request of the State due to the support of other parties for this activity. The project met its revised target of delivering 8,500 long-lasting insecticidal bed nets and utilized the intended funds to provide 25 fogging machines, an x-ray machine, and 2.5 tonnes of insecticide a the request of the Ministry of Health.

(ii) Contributing to capacity building of the state government to enable it to more effectively execute its mandate, especially at the local (decentralized) level. Modest. While the project helped to train over 1,082 local officials in six key Ministries, and supported the construction or rehabilitation and equipping of 65 administrative buildings, there is little evidence provided that, by project closure, the state government was able to more effectively execute its mandate, especially at the local (decentralized level). The training helped to increase knowledge of the subjects and procedures for operating public programs, including eventually the capacity to monitor the programs. However, project documentation does not provide any specific examples of how the training enabled the state or local government to more effectively exercise tis mandate. An exception at the local level is the performance of the water management committees which were established under the project and which are reportedly functioning with income derived from water user fees.

5. Efficiency:

Efficiency is rated modest. The ICR did not provide an assessment of the value for money. Were the inputs contracted at the right price? How well are those inputs converted to outputs - were the water supplies, roads, and other services constructed and supplied with sufficient quality and without waste? Neither did the ICR supply a quality assessment of the project-supported infrastructure, or a discussion of the costs and benefits of capacity building, community engagement, environmental management and similar activities. The ICR provided information only about the overhead costs of delivering services, comparing UN execution with country execution, which is not a relevant barometer of project efficiency.

There were a number of operational and administrative inefficiencies leading to delays in both effectiveness and completion.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project objectives are highly relevant, while relevance of design is rated substantial. Efficacy in achieving the project's service delivery objectives was high. However, more investment in capacity development would have been needed to enable the State Government to execute its mandate more effectively, especially at the local (decentralized level). The efficacy of the second objective is therefore rated modest. Efficiency is rated modest. Overall outcome is assessed as moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The Risk to Development Outcome is high owing to political instability and lack of security. Beginning in June 2011, a new conflict erupted in South Kordofan, triggered by a contested election, and exacerbated by issues left unresolved by the Comprehensive Peace Agreement (CPA). Risks to maintaining the assets and services provided under the project vary and little information is available with regard to the recurrent finance available for their maintenance and operation. The ICR refers to several commitments made by government entities to maintain services (e.g. the Ministry of Physical Planning and Public Utilities has committed to maintaining and improve the state roads and sufficient budget provision will reportedly be made for this purpose) but as of project closure, no evidence was provided to suggest that these provisions had been put in place.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
Although the project was an emergency operation, it took two years to complete the draft project proposal and a further eight months to reach effectiveness. The project document identified several risks, but in so far as the project was scheduled to be implemented over the course of one year, it is not clear from the project proposal how those risks were intended to be mitigated (for example, in emergency operations, aspects of the Bank's safeguard policies can be delayed, but not left out of project implementation). Many of these risks could be managed during supervision because of the project delays. The project preparation period was not used to develop mitigating measures. However, the project was designed to include interventions that were reflective of state and local development plans and priorities (rehabilitation of existing schools, water points, a Vocational Training Center, provision of ambulances, x-ray machines, fogging machines for mosquito control at a health center, and rural livelihoods). It was also designed to include strong community participation and management (water points and the rural livelihoods program) in implementation. Yet such participation was lacking both in design and in measuring and reporting on results.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
The Bank undertook seven supervision missions roughly every six months, including support for fiduciary management and safeguards. The ICR reports that Bank supervision missions supported learning and corrective decision-making that modified planned activities where necessary, for example, by reducing the anti-malaria bed-net component since other donors were providing that support. Bank supervision, especially through the mid-term review, helped the project management staff to overcome the shortfall in counterpart financing, by working with the state government and the Unity Fund to fill the financing gap. Following a country portfolio review, the Bank supervision team also helped the project team institute new monitoring mechanisms, although this was late in the project cycle. Based on the findings of the Country Portfolio Review, the supervision team also secured funding from the Bank's Gender Action Plan to reach vulnerable populations in the Nuba mountains who had been most affected by the civil war. Implementation support also helped to secure regular financial reports and project audits which, according to the ICR, were timely and of good quality.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Government performance - at the national level - is rated Moderately Unsatisfactory owing to the failure to deliver program funding as agreed upon in the Grant Agreement and as part of the Comprehensive Peace Agreement CPA. The lack of counterpart funding caused significant delays. It also risked sending a dangerous signal in this post-conflict setting that there was a lack of commitment towards implementing the provisions of the Comprehensive Peace Agreement, including facilitating fiscal transfers to the three areas most affected by the conflict, and most contested with regard to the peace agreement. However, the state government made additional funding available through the state budget, and access was granted to the Unity Fund Sudan, thereby allowing funding levels to exceed the amount that was planned.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:

While there were several entities involved in implementing this project, two separate implementing modalities were chosen: (1) execution though UN agencies and (2) execution through national and state government entities, including the project management unit. The UN agencies generally performed well in the execution of their activities, meeting or surpassing most targets. The Project Management Unit was responsible for overseeing most of the government-led activities. While the lack of counterpart funds delayed the implementation of activities under the PMU’s control, the Unit played a part in obtaining alternate financing from the state and the Unity Fund to complete its activities. Having little to no experience managing a World Bank operation, project management staff demonstrated commitment to working within its norms and learning its procedures – the PMU provided regular financial reporting and supported the regular auditing of its project accounts. The Project maintained an adequate procurement register and, according to the ICR, was compliant with the recommendations of the safeguard and gender audits conducted later in the life cycle of the project.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
Conceived as an emergency recovery program to be implemented over the course of just one year, the Final Project Proposal did not include a comprehensive monitoring and evaluation system. Monitoring and reporting arrangements were delegated to the various implementing agencies. The project did not create a central nodule for compiling and analyzing the data under various components, flagging emerging issues and aggregating data to track implementation progress. This role was not assigned to the Project Management Unit (PMU) and no monitoring arrangement was established to provide an overall picture of progress over time. In the closing stages of the project however, following the recommendations of the Bank’s Country Portfolio Performance Review, the monitoring unit in the Bank’s Khartoum office developed common reporting formats to which this and all other MDTF projects were to comply.

b. M&E Implementation:
M&E implementation was carried out separately by the respective parts of the program. As stated in the design section, a mechanism for aggregating findings to inform decision-making across parts of the project was lacking. For example, the PMU compiled progress reports only for the four components it was responsible for implementing. It did not maintain a record of performance under the components entrusted to UNIDO and UNICEF. An MDTF Monitoring Agent was expected to make physical inspection of project activities under all MDTF funded projects, but it was not always able to do so due to the workload involved in monitoring 13 MDTF projects.

a. M&E Utilization:
There were several impediments to M&E utilization that stemmed from the weak design of the M&E system described above. While three different agencies produced some type of monitoring report (the PMU, the Bank’s Khartoum office, and the MDTF Monitoring Agent), none of these reports was produced in a timely enough way to be utilized to address emerging implementation issues or to adjust project design. In addition, the entity that was charged with making such recommendations – the State Steering Committee – met too infrequently and is reported to have been ineffective.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
According to the Final Project Proposal (page 25), the only safeguard triggered was Environmental Assessment (OP 4.01). The Final Project Proposal classified the project as Category B, although it is erroneously referred to as Category C in the ICR (page 7) . While the project could be processed under simplified procedures with regard to environmental safeguards policies, due diligence on safeguards was still required during implementation. As noted in the Final Project Proposal, certain components such as the rehabilitation of roads, the construction of culverts and bridges, and the de-mining of villages and roads (safe disposal) were likely to have an environmental impact. Although no Environmental Assessment had been carried out within 12 months of effectiveness (in accordance with Bank Guidelines), the ICR reports that corrective measures were adopted in 2010 following a comprehensive Environmental and Social Management Audit. These measures included mainstreaming environmental considerations across the project, including fencing of hafirs, segregating water arrangements for humans and animals and safe disposal of wastewater. The ICR (pages 6-7) reports that construction and rehabilitation of buildings was executed with due regard to environmental considerations ( the rehabilitation of the Kadugli Vocational Training Center was undertaken within the existing compound of the Center and did not require new land to bring under construction works). However, the ICR contains no clear statement that there was full compliance with OP 4.01.

b. Fiduciary Compliance:
Financial Management

Recognizing that emerging from a conflict situation, the state government and local authorities had weak fiduciary capacity, the Bank entered into direct agreements with the UN agencies and transferred in lump sum the entire cost to each one for implementing its components. It was agreed that each agency would apply its own fiduciary rules in procurement, disbursement and financial management as far as expenditures on the project components were concerned. The Bank task team members, including fiduciary specialists, visited the project area several times, maintained close contact with the implementers, and ensured that interim financial reports were submitted and project accounts audited on time, Audit reports submitted to the Bank were of acceptable quality. The ICR does not indicate whether the audits were unqualified.

Procurement
The project maintained an adequate procurement register that was subject to periodic Bank prior review audits. There were no reported cases of misprocurement.

c. Unintended Impacts (positive or negative):

d. Other:
Gender: The Project commissioned a gender audit at mid-term to assess the extent to which it was gender sensitive and how this could be improved. The recommendations informed both the implementation of the remainder of the Project as well as the Tundia women’s empowerment sub-project described in Section 2c of this Review. Overall, women constituted 41% of the total beneficiaries and in training carried out in the last year of the project, women constituted 46% of all participants. In vocational training, 43% of women were trained at the training institute Kadugli. In water user associations established under the Project, over 30% of the management committee members are women.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Significant
High
Political risks that were heightened in 2011 remain, and the ICR provides no evidence that maintenance and other recurrent costs associated with many of the services and assets delivered by the project will be covered.  
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

There are three lessons from the ICR that can be applied more broadly to the Bank's work in Fragile and Conflict affected settings.

There are tensions and trade-offs between the need to act quickly to stabilize conflict affected populations and longer-term capacity development objectives. Where there is an inherent tension between the need to respond quickly, as in an emergency operation, and the need to build local capacity, project design needs to acknowledge this tension, the potential trade-offs and ways in which these can be resolved during project implementation.

The need to act quickly does not remove the desirability of ensuring some level of stakeholder involvement and consultations, which can be deepened as the project is implemented. The importance of stakeholder consultation at project formulation, during implementation and for project performance evaluation cannot be overemphasized.

Preparing counterpart agencies to deal with Bank procedures and systems may require a more concerted effort a new implementing unit is being established, where capacity is low (which is the case in most conflict and fragile situations), and where government counterparts have not worked with the Bank for a long period (as was the case in Sudan). A detailed Project Implementation Manual, drafted during preparation and finalized after incorporating stakeholders’ suggestions, could be used as a Handbook by the Project Management staff. Another key lesson is the need to provide comprehensive training to the project management staff in Bank procedures at project start-up because of their non familiarity with these procedures. A one day orientation does not serve the purpose.

14. Assessment Recommended?

Yes
Why?
The MDTF Country Portfolio Performance Review was undertaken by the World Bank, the Government and donors when just 4 of the 15 projects had closed. With the closing of the original program on June 30, 2013, a clustered project performance assessment of the different MDTF interventions would be timely.

15. Comments on Quality of ICR:

The ICR is comprehensive and covers all aspects of project implementation. It was able to simplify and explain a rather complex operation with several components and roughly the same number of executing agencies. The assessment of project performance allowed for the constraints posed by a post-conflict environment and the ratings were realistic. The lessons are applicable to other fragile and post-conflict settings. There are, however, some shortcomings. There is no clear statement that the triggered safeguard policy was complied with, or whether external auditors’ opinions were qualified. The project cost table in Annex 1 does not contain a break-down by component

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions