Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Horn Of Africa Emergency Health And Nutrition Project


  
1. Project Data:   
ICR Review Date Posted:
12/09/2013   
Country:
Africa
PROJ ID:
P127949
Appraisal
Actual
Project Name:
Horn Of Africa Emergency Health And Nutrition Project
Project Costs(US $M)
 30.0  30.0
L/C Number:
Loan/Credit (US $M)
 30.0  30.0
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  09/15/2011
 
 
Closing Date
03/29/2013 03/29/2013
Sector(s):
Health (100%)
Theme(s):
Child health (40%) Nutrition and food security (30%) Other communicable diseases (30%)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Moritz Piatti
Judyth L. Twigg Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:


    According to the Emergency Project Paper (pp. 4-5), the Project Development Objective (PDO) was: “to support the emergency response in targeted refugee camps, in Kenya and Ethiopia, by expanding implementation of a health and nutrition package of services, in a manner consistent with the sub-region’s medium-term human development goals.”

    According to the Grant Agreement (Annex B, p. 13), the objective of the Project was: “to support the emergency response in Targeted Refugee Camps (as defined below) in the beneficiaries’ territory by expanding implementation of a health and nutrition package of services, in a manner consistent with the sub-region’s medium-term human development goals."

    The Grant Agreement is used for this Review, as it is the legally binding agreement, and the objective statements are almost identical.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

Component 1 (appraisal US$ 27.9 million, actual US$ 27.2 million): Treatment and prevention of malnutrition and provision of basic health services, including the screening of beneficiaries and direct costs associated with service delivery. This component was to expand the coverage of the United Nations High Commissioner for Refugees' (UNHCR’s) core package of health, nutrition, and population-related services being provided in refugee camps in Dadaab, Kenya and Dollo Ado, Ethiopia. It included several sub-components:

(a) Treatment and prevention of malnutrition among vulnerable populations, particularly children and women. This sub-component was to address the prevention and treatment of acute malnutrition among children, in addition to focusing on the nutritional needs of pregnant and lactating (P&L) women.

(b) Screening and provision of basic health services. This sub-component was to focus on maternal and child health services, in addition to the prevention and treatment of common sources of morbidity and mortality, including respiratory tract infections, diarrheal disease, and malaria.

(c) Strengthening of sanitation facilities and safe water supply as key contributors to improved health status. This sub-component was to enable the provision of improved water and sanitation facilities in the targeted areas.

(d) Strengthening project monitoring and evaluation (M&E). This sub-component was to support M&E activities (including the innovative use of ICT technology to facilitate data collection, monitoring, and management).

Component 2: Project Management (appraisal US$ 2.1 million, actual US$ 2.0 million). Project management was to consist of indirect costs required to support project delivery.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost: In Special Drawing Rights (SDR) terms, 100% of the grant was disbursed.

Financing: The project was financed by an International Development Association (IDA) grant in the amount of US$ 30.0 million. Bank funds were disbursed into a pool financed by multiple donors. The Bank share of the pool was approximately 12% - 20%, but precise estimates are unavailable because of non-cash food donations from the United States government. The operation took place under the Fiduciaries Principles Accord (FPA), outsourced to UNHCR, and thus viewed through a different lens than a standard IDA investment project. All Bank funds transferred were received, administered, managed, expended, reported on, and audited in accordance with the policies and procedures of UNHCR; however, the FPA includes specific provisions to ensure that Bank standards are explicitly upheld (ICR, p. 5).

Borrower Contribution: There was no Borrower contribution.

Dates: The expected effectiveness date was 09/30/2011, and actual effectiveness was 09/19/2011. The project closed as scheduled on 03/29/2013. There were no extensions or delays.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

High. The development objectives of supporting the emergency response in targeted refugee camps, in Kenya and Ethiopia, by expanding implementation of a health and nutrition package of services, in a manner consistent with the sub-region’s medium-term human development goals, were and remain highly relevant to country conditions and to Government and Bank strategy.

In 2011 the Horn of Africa experienced its worst drought in 60 years, affecting over 12 million people and reversing progress in the reduction of poverty, disrupting food production systems, and jeopardizing hard-won improvements in human development. Due to the persistence of food insecurity and amplified by conflict, by mid-2011 an estimated 25% of Somalia’s population had been displaced, with many seeking refuge in neighboring Ethiopia or Kenya. With a daily arrival rate of over 2,000 refugees, camps were overwhelmed, and the region, which was already suffering from a food security emergency situation, was exposed to additional stress.

With the onset of the drought, the Bank engaged in an Emergency Ministerial-Level Meeting on the Horn of Africa, where there was consensus that relief and recovery were at the first order of priority, followed by medium- to long-term support to promote economic recovery and drought resilience. The Country Partnership Strategies (CPS) between the Bank and both countries contain a strong focus on human development, disaster management, and managing risks. In Ethiopia this is reflected in the CPS 2012, which has "Enhanced Resilience and Reduced Vulnerabilities" as a separate pillar with explicit targeted outcomes such as: improved access to quality health services; enhanced resilience of vulnerable households to food insecurity; increased adoption of disaster risk management systems; and sustainable natural resource management and resilience to climate change (Ethiopia CPS, pp. 66-69). Similarly the results framework of Kenya’s 2010 CPS includes relevant objectives such as: better access to health care, education, and basic infrastructure services; and improved management of key natural resources (Kenya CPS 2010, pp. 43-46). Additionally, one of Kenya’s Vision 2030 Medium Term Plan Objectives is to reduce losses due to floods, droughts, climate change, and desertification (Kenya Vision 2030, p. 137).

b. Relevance of Design:

Substantial. The project's results chain is tightly designed and highly plausible. Treatment and prevention of malnutrition and the provision of basic health services in a refugee camp is highly likely to translate into improvements in severe acute malnutrition. Preventive activities such as routine and mass vaccinations and growth monitoring are conceivably correlated with improved child health. Similarly, it is highly plausible that strengthening of sanitation facilities and safe water supply acts as a key contributor to improved health status.

The project's design is based on best practice in emergency refugee response appropriate for the drought context, and included lessons from previous operations in the Horn of Africa, as well as from global disaster risk management and emergency recovery/reconstruction experience. The choice of UNHCR as an implementing agency was also built on collaboration experience with UNHCR in the Horn of Africa and the Great Lakes Region.

The project's objective was to strengthen immediate emergency relief only. Medium- to longer-term goals were beyond the scope of the project.


4. Achievement of Objectives (Efficacy) :

To support the emergency response in targeted refugee camps, in Kenya and Ethiopia, by expanding implementation of a health and nutrition package of services is rated Substantial.

World Bank funds were disbursed into a pool financed by multiple donors and managed by UNHCR. The Bank's share was approximately 12% - 20% of the total, but precise estimates are unavailable because of substantial non-cash donations from the US Government (TTL interview).

Targets were calculated based on the Bank grant and the anticipated refugee population. Actual numbers were reported on the activities funded by the entire pool, and it is not possible to isolate Bank funds from the general pool. The Bank is therefore one of multiple contributors to the pool's achievements. All targets were drastically surpassed because many more refugees arrived than originally anticipated, and funding to the pool increased from other donors (TTL interview).

Outputs:

406,895 children 6-59 months old and pregnant/lactating women were screened, far exceeding the target of 123,475.

85,967 children 6-59 months old with severe acute malnutrition were referred for treatment / food supplements, far exceeding the target of 5,275.

174,403 pregnant and lactating women received food supplements, far exceeding the target of 23,475.

27,239 latrines were built or renovated for improved sanitation services, far exceeding the target of 5,000.

Over 32,000 treated bed nets were distributed in Dollo Ado, Ethiopia and over 161,000 in Dadaab, Kenya. Of these, 88,000 were supported by IDA.

The total number of direct project beneficiaries was 1,666,824, far exceeding the target of 440,272.

Intermediate Outcomes:

The number of children under five years of age treated for severe acute malnutrition was 85,967, far exceeding the target of 5,275. In Dollo Ado, Ethiopia over 89.2% of children were enrolled in severe acute malnutrition programs by March 2013, almost meeting the standard of 90%, No target was set.

The number of pregnant and lactating women who received food or micronutrient supplements was 173,541, far exceeding the target of 23,475. In Dollo Ado, Ethiopia, coverage rates under the supplementary feeding program targeting pregnant and lactating women progressively improved from 44% to 97% throughout the project reporting period. No target was set.

The number of children under five years who received treatment for acute respiratory infections was 209,466, far exceeding the target of 68,263.

The number of children under five years who received treatment for watery diarrhea was 88,939, far exceeding the target of 23,662.

77,238 targeted children 9-59 months old were vaccinated against measles, exceeding the target of 70,000.

70,000 targeted children 6-59 months old received one dose of vitamin A supplement, meeting the target of 70,000.

Most coverage rates as a share of the total refugee population are not reported due to the transient nature of the camps, with refugees arriving and leaving daily.

Outcomes:

In Dollo Ado, Ethiopia, the recovery rate of children aged 6-59 months suffering from severe acute malnutrition increased from 68.8% in March 2012 to 84.5% in March 2013. No target was set. The standard recovery rate is 75% (ICR, p. 14).

In Dollo Ado, Ethiopia, the mortality rate from severe acute malnutrition of children aged 6 - 59 months was kept at 0.3%, considerably below the standard of <10%. No targets were set.

In Dollo Ado, Ethiopia recovery rates for moderate acute malnutrition improved from 85.6% in September 2012 to 92.7% by March 2013. No target was set.

In Dollo Ado, Ethiopia, the incidence per 1000 of acute watery diarrhea among children under five years declined from 23.5% in March 2012 to 18% by March 2013. No target was set.

No outcome indicators are available for the Dadaab camp in Kenya. The project supported a short-term humanitarian crisis response, and this is clearly reflected in the Project Development Objective (PDO). Given the short implementation period (18 months), a conscious choice was made to use only service delivery indicators in the Results Framework. Given the lag between many health outcome improvements and services delivered, this was an appropriate approach for the project’s context. The interventions are justified by referring to academic literature, which shows that short-term actions to address urgent health and nutrition needs – particularly among infants and children – have significant medium- to longer-term effects on cognitive development, growth, and nutritional status and thus human capital development.


5. Efficiency:

Efficiency is rated as high.

The Bank does not conduct appraisal of specific activities under the Fiduciary Principles Accord (FPA). In place of a formal economic and financial analysis, which was not feasible, a detailed contextual country analysis was conducted for Ethiopia and Kenya. The assessment supported: (i) the rationale for delivery of health and nutrition services in the targeted camps; (ii) the activities under the UNHCR program as appropriate to respond efficiently to the needs on the ground; and (iii) the UNHCR’s implementation structure as conducive to cost-efficient provision of services, given the challenging operating environment.

The implementation of planned activities was done in a timely manner. The project supported rapid expansion and strengthening of UNHCR’s ongoing health program, was built on existing infrastructure and systems, and worked through strong, efficient specialized UNHCR partner institutions. UNHCR has been coordinating service delivery in the Dadaab refugee camp for over 20 years and in Dollo Ado for nearly 15 years. UNHCR has well-established supply chains and was able to leverage economies of scale in procurement that would not have been possible through a separate stand-alone project. Medical supplies and other related commodities financed by the project were procured directly through UNHCR’s established global supply chains and benefited from bulk procurement pricing. In addition to the lower costs associated with such bulk procurement, this arrangement also enabled critical standards regarding both quality and reliable/rapid delivery to be met. The implementation efficiency of this project benefited from this extended service delivery experience in Dadaab and Dollo Ado, which also enabled the mitigation of risks resulting from the challenging geopolitical environment (UNHCR has substantial experience in planning for, and rapidly responding to, service delivery challenges caused by annual/seasonal flooding in Dollo Ado; similarly, UNHCR has long-term expertise in addressing periodic security challenges around Dadaab's camp perimeter, in collaboration with national authorities). Such extended experience in planning/managing camp operations in Dadaab and Dollo Ado enabled consistently strong service delivery, and rapid scale-up, as is evidenced in each of the six monthly technical reports.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


The importance of the project given the region’s circumstances and sector priorities, in combination with a plausible results chain, renders the relevance of objectives high and the relevance of design substantial. The efficacy for the objective is rated substantial, given that all indicators well surpassed their target values, and these efforts can plausibly be expected to translate into improved outcomes. Efficiency is rated high due to implementation of activities in a timely manner and economies of scale accruing through the use of UNHCR as an implementing agency that would not have been possible through a stand-alone project.


a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:


Given the volatile political and security environment in the Horn of Africa, Risk to Development Outcome is rated high. Challenges in sustaining critical services within Dadaab and Dollo Ado persist with continued new arrivals to the camps and anticipated extended-term needs of the Somali refugee population. While the Bank plans to continue to support the governments of Kenya and Ethiopia with broader development needs, the project itself does not accommodate for a longer-term engagement to assist with drought resilience, regional stability, or absorption capacity of the refugee camps.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:

As part of OP/BP 8.00, "Rapid Response to Crises and Emergencies," including the request for the Board waiver and eventual Board approval of the FPA application, the Bank took measures to assess quality at entry by an Operations Policy and Country Services (OPCS) review conducted at the vice presidential level.

The Bank responded rapidly to an emergency situation. The operation was prepared by a multi-disciplinary Bank team under exceptional circumstances in close collaboration with UNHCR. The design included appropriate objectives and indicators and a clearly outlined results chain and results framework. Well-planned monitoring and evaluation arrangements, in agreement with UNHCR, took advantage of existing information systems and ensured the availability of reported results. Lessons from previous operations in the Horn of Africa, as well as from global disaster risk management and emergency recovery/reconstruction experience, were taken into account. The operation also built on collaboration experience with UNHCR in the Horn of Africa and the Great Lakes Region. Furthermore, a background analysis, conducted in close collaboration with the United Nations Office for the Coordination of Humanitarian Affairs (UNOCHA) and UNHCR, informed the design, which was subsequently endorsed by counterparts in Ethiopia and Kenya.

In addition, the project was designed to enable retroactive financing of up to US$ 3 million, which gave UNHCR the ability to initiate urgent procurement and other related transactions during the period immediately preceding the Board date. This allowed UNHCR the ability to scale up support almost immediately, which was especially critical at the peak of the crisis.

Quality-at-Entry Rating: Highly Satisfactory

b. Quality of supervision:

Operations under the FPA are viewed through a different lens than standard IDA investments, since the project was essentially outsourced to UNHCR. In July 2012, a World Bank mission, accompanied by UNHCR representatives from Geneva and Addis Ababa, visited Ethiopia’s Dollo Ado refugee camps. As a result of the security situation, this was the only Bank mission conducted over the 18 month implementation period.

Targets were drastically surpassed, which indicates a degree of conservative target setting. While these targets were set at the design stage based on available knowledge while the crisis was unfolding, they could have been revised during the course of implementation to more accurately reflect the magnitude of the rapidly increasing health and nutrition needs of the refugees.

Throughout implementation, UNHCR submitted comprehensive technical progress reports that reported on the project’s results framework and included available data on camp population and statistics and various survey results. Complete data were reported in the ISRs for all PDO and interim indicators for each 6-month period of project implementation. In addition, UNHCR provided progress reports every six months as agreed, which included detailed information on activity implementation progress, challenges and resolution of bottlenecks, as well as data collected and analyzed per camp. The reports also included detailed annexes that contained, for example, camp population statistics, results from joint nutrition and health surveys, water/sanitation/hygiene (WASH) camp assessments, and a knowledge, attitudes and practice survey.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Governments of Ethiopia and Kenya showed strong commitment to addressing the crisis affecting their countries, and both endorsed the decision to outsource implementation to UNHCR. There was no interference from governments or additional requirements, restrictions, or limitations put in place that would hinder UNHCR’s implementation of the project. However, there were no direct Government contributions to UNHCR. In Kenya, the Ministry of Health (MOH) is overseeing disease surveillance in the camp.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:

Given the complexities and challenges of oversight and implementation in two separate populations separated by country borders, UNHCR was well suited for the task, as originally expected. It worked to achieve impressive results in a challenging environment and reported on a timely basis in great detail. No major problems of misprocurement or mismanagement of funds were reported. Technical and financial reports were submitted on time and were complete, with both quantitative and qualitative data collected by UNHCR and reported against this operation’s results framework. Subcontracted NGOs with a long standing track record working with refugees carried out activities on a timely basis, under huge demands, with positive results.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

Overall, the project design included clear and pragmatic M&E arrangements. The emergency project paper presented a strong results framework, contained a clear results chain to delineate links between project inputs and targeted outcomes, and elaborated a plan for data collection and monitoring arrangements. The M&E data collection was based on UNHCR’s existing health information system, which provided timely data validated by its field presence.

The selection of indicators in the Results Framework was guided by the content of the package of services provided by UNHCR in the camps. Indicators were selected from those that were already collected by UNHCR in both refugee camps to ensure minimal additional data collection burden and disruption to service delivery. The quality of the indicators was also assessed according to accepted criteria such as clarity, relevance, adequacy, feasibility, and economy of implementation. However, the project would have benefited from reporting the share of people covered rather than the absolute number of beneficiaries.

Given the relatively short implementation period (18 months), a conscious choice was made to use service delivery indicators in the Results Framework. Given the lag between many health outcome improvements and services delivered, this was a pragmatic approach for the project’s context.

b. M&E Implementation:

UNHCR reported on a timely basis in great detail despite the challenging environment. Comprehensive technical progress reports were submitted on time on the project’s results framework and included available data on camp populations, statistics, and various survey results. Complete data were reported in the ISRs for all PDOs and interim indicators for each 6-month period of project implementation, including detailed information on activity implementation progress, challenges, and resolution of bottlenecks as well as data collected and analyzed per camp. The reports also included detailed annexes which contained, for example, camp population statistics, results from joint nutrition and health surveys, WASH camp assessments, and a knowledge, attitudes and practice survey.

a. M&E Utilization:

The project used M&E data generated to assist in planning and guiding the numerous operational assessments conducted by UNHCR. This enabled the project to be more responsive to the evolving needs of highly vulnerable sub-groups within the refugee camps (particularly with respect to the service delivery needs of new arrivals from Somalia).

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:

Given the application of the FPA for this operation, World Bank safeguard policies did not apply. UNHCR's 2005 Environmental Guidelines were adopted for this operation according to UNHCR's standard project implementation mechanisms. The UNHCR Guidelines were assessed as largely consistent with the Bank’s respective policies. The project was classified as a Category B operation, and OP 4.01 (Environmental Assessment) was triggered for the purpose of Bank records and to signal that had this been a standard Bank operation, not using the FPA instrument, these elements would be appropriate and relevant safeguard tools would have been prepared. An Integrated Safeguards Data Sheet was also completed.

b. Fiduciary Compliance:

Fiduciary issues were handled by the implementing agency, UNHCR. As agreed under the FPA, the financial management and disbursement procedures of UNHCR applied. UNHCR maintained a separate ledger account (Grant Control Account) to record the financial transactions of this project. All six-month financial reports (IFR) were submitted on schedule and cleared by the financial management unit of the Africa Region (AFTFM). The Bank's final financial management review indicated that the Statement of Sources and Uses of Funds presented a fair and reasonable accountability of IDA resources channeled to the project since project effectiveness. The project was consistently rated Satisfactory for Financial Management throughout implementation.

c. Unintended Impacts (positive or negative):

The refugee camps targeted in the project serve and continue to serve as centers for provision of health, nutrition, and sanitation services. The project supported such interventions for the target population, but the surrounding communities also benefitted from access to services, possibly indirectly raising the bar in terms of access and quality of services available for groups in remote areas. This avoided distortions in the quality and level of services available and supported the national governments as they faced enormous challenges in addressing the rapidly evolving and increasing needs of refugees occupying their territories.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
High
High
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

UNHCR as an implementing agency of an emergency relief project can generate substantial efficiency gains. A project can benefit from UNHCR's well - established supply chains and leverage economies of scale in procurement that would not be possible through a separate stand-alone project.

The possibility of retroactive financing can encourage and enable rapid implementation start-up in an emergency response context.

This operation’s Board waiver enabled substantial policy exceptions to standard IDA lending terms, which granted the operation substantial flexibility. An emergency response project can benefit significantly from such flexibility in terms of rapid preparation, efficient implementation, and achieving results.

Support to refugee camps may entail important spillovers to nearby communities. An emergency response project could benefit by anticipating such during project design.


14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR provides a concise, clear assessment of the project's achievements that complies with established guidelines. It places the project thoroughly into context, and provides an extraordinary level of detail on how the project fits into the larger development agenda of the Region and the Bank. The ICR is careful in analyzing the results framework, and provides valuable information on the reasons why specific indicators were chosen.

However, the ICR would have benefited from additional information on other partners that were engaged in the emergency relief efforts and an analysis of why the indicators were surpassed so drastically. The Bank's contribution as a share to total emergency relief funding would also be helpful information to inform attribution. In addition, reporting the coverage of intervention beneficiaries (rather than the number only) would provide additional insight.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions