Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Avian Influenza Control And Human Pandemic Preparedness And Response Project


  
1. Project Data:   
ICR Review Date Posted:
06/17/2013   
Country:
Kosovo
PROJ ID:
P102165
Appraisal
Actual
Project Name:
Avian Influenza Control And Human Pandemic Preparedness And Response Project
Project Costs(US $M)
 3.0  3.1
L/C Number:
CH272
Loan/Credit (US $M)
 3.0  3.1
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  02/01/2007
 
 
Closing Date
11/29/2010 06/30/2011
Sector(s):
General public administration sector (50%), Health (28%), Agricultural extension and research (19%), Solid waste management (3%)
Theme(s):
Other communicable diseases (25% - P) Natural disaster management (25% - P) Rural services and infrastructure (24% - P) Pollution management and environmental health (13% - S) Health system performance (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Stephen Hutton
Roy Gilbert Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
Original objective:

The original project objectives were to "strengthen the PISG’s [Provisional Institutions of Self Government] capacity to prevent the spread of Highly Pathogenic Avian Influenza (HPAI) among poultry, to prevent the transmission of avian influenza from birds to other animals and humans, and to prepare for a potential pandemic of avian influenza transmissible between humans." (Financing agreement, Schedule 1).

The objectives in the appraisal document (Technical Annex, page 7) were substantially identical.

Revised objective:
After restructuring, the revised objectives were to "minimize the threat posed to humans by highly pathogenic avian influenza (HPAI) infection and other zoonoses to prepare for, control, and respond to influenza pandemics and other infectious disease emergencies in humans". (Project Paper December 18 2009 that restructured the project).

The revised objectives were largely similar to the original objectives, except that they expanded the scope to cover zoonoses other than avian influenza, and pandemics and infectious disease emergencies other than those from avian influenza.

The Provisional Institutions of Self Government were the administrative government set up in Kosovo by the 1999 United Nations Security Council Resolution 1244, which ended the conflict with Serbia and establishing Kosovo as a self-governing entity. Kosovo later declared full independence in 2008.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 12/18/2009

c. Components:
Original components (and their restructuring):
1. Public awareness ($0.28 at appraisal, $0.24m after restructuring, $0.26m final), aimed at developing and implementing a communication plan to inform the public (particularly those at high risk) about the threat of avian influenza and steps they could take to reduce transmission and spread. It would include development and dissemination of communication materials, communications training, and awareness raising with community groups. After restructuring, the communication materials were expanded to also cover the 2009 influenza pandemic.

2. Animal health ($1.47m at appraisal, $1.00m after restructuring, $1.25m final). This included support for prevention and preparedness capacity, disease control capacity, surveillance and diagnostic capacity, and outbreak containment and control capacity. It would include policy reform, technical assistance for veterinarians, epidemiological studies, laboratory upgrades, equipment and training for culling, and a compensation fund for farmers whose birds were culled. After restructuring the money for the compensation fund was removed, additional civil works and equipment for the animal health laboratory were added to ensure that the laboratory would be operational, and additional technical assistance on field epidemiology and laboratory diagnosis for other zoonoses were added.

3. Human health ($0.97m at appraisal, $1.49m after restructuring, $1.31 final). This included support for coordination and planning, improvements to the national public health surveillance system, and improving heath care response capacity. It included creation of plans, technical assistance and training to set up a surveillance system, laboratory upgrades, treatment facility upgrades, and supply of vaccine and antivirus. After restructuring, additional activities were added to revise the National Pandemic Preparedness Plan, to support hospital preparedness for the 2009 influenza pandemic, to purchase additional protective clothing and laboratory consumable items, and to support additional technical assistance for emergency management, triage, and clinical management.

4. Project management ($0.28m at appraisal, $0.27m after restructuring, $0.29m final). This would cover project support, training, and M&E.

The project restructuring kept most existing activities in place, but shifted funds from the compensation fund to other activities in the public awareness, animal health, and human health components as described above.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
As the project used an emergency lending instrument, the operation was to be completed in three years rather than the usual five years. The difference between the US$ value of the grant at appraisal ($3m) and the actual project cost ($3.1m) is driven by exchange rate movements (the grant was denominated in special drawing rights).

The project was entirely financed by an IDA grant of SDR 2 million (US$3 million equivalent). There was no borrower contribution, though government funds were made available to be used for compensating farmers whose birds were culled if necessary.

The project closure date was extended twice, to December 31 2010 and then to June 30 2011. These extensions were made in order to allow time for the completion of activities added under the project restructuring and reallocation.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Relevance of original objectives: Substantial
The project was responding to a global emergency, rather than being motivated by existing Bank or national development strategies for Kosovo. Kosovo was not a formal World Bank Group member until FY09, so no Country Partnership Strategy existed prior to FY12. The project was not particularly relevant to the pillars of interim strategy notes during the project period (it was tangentially relevant to a goal to "create strong, transparent institutions"), but was consistent with the broad World Bank approach of responding to highly pathogenic avian influenza (HPAI).

The original objectives addressed HPAI, which posed a serious risk to Kosovo. The disease was having a significant effect on the poultry industry in many countries, but there was a particular risk for Kosovo as outbreaks had occurred among poultry or wild birds in several nearby countries (Bosnia and Herzegovina, Bulgaria, Croatia, Serbia and Montenegro and Romania). The virus could spread to Kosovo through migration of wild birds or through illegal imports of poultry.

Kosovo faced significant vulnerabilities from avian influenza. Kosovo had 2.6 million poultry, of which 75% were on small rural farms with modest biosecurity. Half of all rural households owned poultry, so there was a significant human population that was potentially vulnerable to exposure to disease transmission from birds.

Veterinary and public health services had a number of weaknesses in capacity, public awareness and ability to fund compensation. Though the Kosovo government had set up HPAI-related plans and UN agencies and donors had provided some support, shoring up these weaknesses was beyond the financial capacity of the government or donors.

The vulnerabilities to avian influenza and the low capacity level meant that the project objectives were relevant for Kosovo. However, the focus on avian influenza was arguably too narrow, particularly in regards to risk of influenza pandemics, which could originate from avian influenza or from other strains of influenza.

Relevance of revised objectives: Substantial
The original objectives were largely encapsulated within the revised objectives, and so the reasons for relevance of the original objective also applied to the the revised objectives.

By 2009, no outbreaks of avian influenza had occurred, and there was a perception that the risk of avian influenza was declining throughout the region and the world. A pandemic of non-avian H1N1 influenza began in 2009, and posed a potential threat to Kosovo.

The revised objectives added attention to other zoonoses, and clarified the objective so as to address non-avian influenza pandemics. These helped the project to remain relevant and to address the unfolding pandemic.

b. Relevance of Design:
Relevance of design to original objectives: Substantial
The original project design followed the broad outlines of the Global Program on Avian Influenza. The awareness and information component was designed to communicate risks to the public and to reduce behaviors that might lead to disease transmission from poultry to humans or between humans. The animal health component was designed to improve the ability to identify any avian influenza outbreaks that occurred among poultry and to contain them by culling infected and potentially infected birds, thus limiting the spread of disease. The human health component was designed to increase the capacity to identify and treat humans infected by HPAI, and to build capacity to mitigate a pandemic. However, the project design did little to address biosecurity and thus chance that an outbreak would occur.

Relevance of design to revised objectives: Substantial
The activities supported under the original project design were still in the revised design, and remained relevant to the revised objectives. The newly added activities supported the revised objectives. Additional improvements to the animal health laboratory would contribute to the ability to diagnose avian influenza and other zoonoses. Purchase of consumable supplies for the human health component, revised communication messages and pandemic preparedness plan and additional hospital preparedness would contribute to increasing capacity to respond to the influenza pandemic. Removing the project funds dedicated to compensation did not undermine the objectives, because the likelihood that they would be needed in the short term had declined and because the government had committed to fund compensation as needed. However, the extent to which new activities addressed zoonoses other than avian influenza or infectious disease emergencies other than influenza is unclear.


4. Achievement of Objectives (Efficacy) :

Achievement of original objectives:
1. Strengthen capacity to prevent the spread of avian influenza among poultry and to prevent the transmission of avian influenza from birds to other animals and humans: Substantial
Outputs:
  • An animal health laboratory with the capacity to diagnose avian influenza and other animal diseases was completed in September 2010, and training was provided in March 2011.
  • A compensation mechanism was established, under which farmers would be compensated for the loss of any poultry culled in order to contain avian influenza outbreaks. Funds for compensation were available throughout the project, initially from project funds and then from the government budget after project funds were reallocated. However, no outbreaks occurred and so compensation funds were not drawn on.
  • A mobile incinerator, disinfection equipment, protective gear and other materials were procured and distributed. These would have assisted in controlling outbreaks had any occurred.
  • An epidemiology unit was established and trained and provided with transport.
  • A public awareness campaign was produced targeting poultry producers, veterinary and health personnel and the general public, aimed at informing people of the threat of HPAI and measures they could take to protect themselves.
  • It should be noted that project activities in areas with a majority Serb population were carried out and financed by WHO and UNICEF, not by the Bank-financed project.

Outcomes:
  • No outbreaks of avian influenza occurred in Kosovo during the project period. It is not possible to assess the extent to which the project contributed to this.
  • Diagnostic capacity was established in Kosovo by the end of the project, but external assessment of the quality diagnostic capacity had not been done by project closure. The ICR did not contain evidence on the effectiveness of surveillance systems.
  • Simulation exercises were carried out to test the readiness of systems for disease reporting and control, but the results of these simulations and the level of preparedness demonstrated was not reported in the ICR.
  • Outputs that prevented the spread of avian influenza among poultry would also serve to reduce the risk of transmission from birds to humans.
  • Knowledge, Attitude and Practice surveys were conducted in 2008 and 2010, but no baseline survey was carried out (understandably given the emergency nature of the project), so survey responses cover two points both after the communication campaign had commenced. The surveys provided mixed evidence of behavioral change. 80% of the general public reported in 2008 that they changed the manner in which they handled poultry or poultry products, but this had fallen to 50% by 2010; this may have represented declining sensitivity to avian influenza as the perceived risk declined. 22% reported cooking poultry well in 2008, rising to 31% in 2010. There was no significant change in the proportion reporting they avoided contact with dead poultry and wild birds (32%, 33%). The proportion of the population reporting they did not consume poultry (an unnecessary precaution) declined from 21% in 2008 to 10% in 2010. 24% of the population could identify key symptoms of avian influenza. The survey found that healthcare staff had not significantly improved their understanding of avian influenza, and that some farmers reported some improvements in biosecurity on their premises while remaining skeptical that an avian influenza epidemic would occur. Given the declining values of some indicators, the survey evidence does not provide much evidence that the communication campaign was effective.
  • Other than some effects from the awareness campaign, the project had little impact on poultry biosecurity.

2. Prepare for a potential pandemic of avian influenza transmissible between humans: Modest
Outputs:
  • A pandemic preparedness plan was developed. A crisis management training module was not completed.
  • Health staff were trained, but the ICR does not specify how many.
  • Surveillance guidelines and procedures were adopted, and an early warning system was established. Transport procedures were established for international testing of specimens.
  • A virology laboratory was upgraded in 2010, and provided the ability to diagnose avian influenza. International testing confirmed the accuracy of diagnostic capacity at at least 95%.
  • 8,000 doses of seasonal influenza vaccine were procured and distributed.
  • An Intensive Care Unit (ICU) was established at Pristina Infectious Disease Hospital.
  • 25,000 doses of antivirus were procured and were ready for treatment.

Outcomes:
  • The early warning surveillance system might have increased the ability to identify and respond to influenza outbreaks, but the ICR does not provide evidence on the effectiveness of the surveillance system.
  • Providing diagnostic capacity within Kosovo has decreased the time needed to receive a diagnosis.
  • The project vaccinated medical staff during the project period but this seasonal influenza vaccine would have had little impact on protecting against avian influenza, and the project did not establish a sustainable system for procuring vaccine.
  • The enhanced ICU increased the ability to treat human cases of avian influenza, though it is likely that a single ICU would be overwhelmed in the event of a serious pandemic.
  • The antivirals purchased would have enhanced avian influenza pandemic preparedness during period of the project, but the doses will expire in the new future and so provide no long-term coverage. It is unclear from the ICR whether these doses were used during the 2009-10 pandemic; if no doses were used despite an influenza pandemic, then the stockpile likely had little value.

Achievement of revised objectives:
Outputs and outcomes of the original objectives were also relevant to the revised objectives.
1. Minimize the threat posed to humans by highly pathogenic avian influenza and other zoonoses
: Substantial
Outputs:
  • The ICR notes (page 23) that the animal health laboratory was capable of diagnosing animal diseases other than avian influenza, but does not specify those diseases or make clear whether reagents, equipment and training were capable of supporting diagnosis for animal diseases other than influenza.
  • Support for the epidemiology team and some equipment would be relevant for increasing capacity to prevent zoonoses other than avian influenza.
  • The ICR notes that the communication campaign was revised to provide information related to other zoonoses.

Outcomes:
  • The outcomes of the original avian influenza objective remained relevant to the revised objective.
  • There is little evidence of the effects of the project on zoonoses other than avian influenza.
  • The impact of the revised communications campaign was not assessed.

2. Prepare for, control and respond to influenza pandemics and other infectious disease emergencies in humans: Modest
Outputs:
  • The preparedness plan covered both avian influenza and other infectious diseases.
  • The additional supplies would help to replace doses consumed during the pandemic.

Outcomes:
  • The availability of antivirals increased preparedness for non-avian influenza pandemics and some other infectious diseases.
  • The intensive care unit upgrades increased preparedness for a wide range of infectious disease emergencies. During the project, the ICU was used in treatment of 30 patients, mostly cases of H1N1 influenza during the pandemic. Four of these patients died, which is within the expected mortality range.

5. Efficiency:

An economic analysis conducted at appraisal attempted to estimate the benefits of the project from reducing the likelihood of outbreaks among poultry and associated losses in poultry. The analysis calculated an internal rate of return of 18%, but this required highly speculative estimates on the efficacy of project interventions and the likelihood of outbreaks. The analysis did not consider the economic benefits from preventing secondary economic costs of outbreaks (such as through a decline in poultry prices driven by fear) or human mortality and morbidity.
The ICR correctly identified that an ex-post rate of return calculation was unlikely to be meaningful, given the extreme difficulties in making reasonable assumptions and given that the project was designed to build capacity to respond to a disaster that turned out not to occur (but was a reasonable fear at the time). But it noted that the project is likely to have had significant benefits from increased preparedness for influenza outbreaks.

The use of project coordinators from existing government structures rather than a standalone project management unit reduced the financial cost of the project, but also contributed to implementation problems, as the coordinators had substantial workloads from their existing jobs. Supervision missions used local experts which reduced supervision costs.

The preparation and supervision costs of the project were high relative to the $3.1 million project cost, with total staff time of 119 staff weeks and $467,000 of travel expenses. This calls into question whether the Bank's intensive project preparation and supervision requirements are appropriate for very small lending operations.

Project efficiency is rated Modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
18%
50%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project was largely carried out according to its design. When no outbreak of avian influenza had occurred by 2009 but a (non-avian) influenza pandemic had begun, the project appropriately broadened its scope to provide some support for the pandemic. The relevance of objectives of the original and revised objectives and design were rated substantial. The project made substantial progress in boosting the capacity of the animal and human health sectors to identify and respond to avian influenza, but there is little evidence on the effectiveness of improvements in preparedness for influenza pandemics and so efficacy of this objective is rated modest. Though traditional economic analysis would not be meaningful for the project, the project may have had significant economic benefits. There were no obvious sources of inefficient use of resources in project supported activities, but Bank supervision and travel costs were very high relative to the size of the supported activities, so project efficiency is rated modest. Together, these lead to a Moderately Satisfactory overall project outcome rating.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

A high level of government commitment through implementation bodes well for the future. Good working relationships achieved between the human and animal health sectors developed during project implementation are likely to continue.

High levels of ownership by the implementing ministries is likely to contribute to project sustainability. The Food and Veterinary Agency and the Institute of Public Health are expected to maintain capacity gains made by the project. Public awareness should be sustained by the Ministry of Health communications working group. Salaries for a team of animal health epidemiologists was included in the budget for 2011 and the planned budget for 2012. The Borrower's Report to the ICR (ICR page 41-42) notes that this will be sufficient to cover operational needs including reagents and training for both animal and human health sectors.

It is unclear whether a mechanism and budget has been established that will refresh the stockpile of antiviral pharmaceuticals as the purchased doses expire.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
The Bank team prepared the project relatively quickly, over four months, drawing heavily on the Global Program on Avian Influenza design template. Preparation included representatives from UNICEF, WHO and FAO, which helped to encourage good cooperation with these agencies during implementation. It turned out that legal requirements for contractual agreements with WHO and UNICEF proved cumbersome and unworkable so were resolved by local agreement.

The project design tried to mitigate key risks. For example, it addressed the potential difficulties in implementing the project in enclaves populated by ethnic minority groups where government activities may not be welcome by supporting outreach through the WHO and UNICEF rather than the project implementing agencies.

One weakness of the design was the decision to use component coordinators who were employed by existing agencies and had existing responsibilities without ensuring that they would have sufficient time to devote to the project. An excessive workload for these staff led to delays in implementation.

The project design did not clearly specify some activities, such as the appropriate specifications for a veterinary laboratory. Discussions on whether the veterinary lab should meet biosecurity level 2 or level 3 caused delays.

There was some discrepancy between the use of an emergency instrument and the capacity-building nature of the project (focusing on laboratory upgrades and civil works that would be slow to implement). But the project's logical design was in part as a contingency mechanism, so that it could have responded to avian influenza outbreaks had these occurred. There was a reasonable fear at the time of design that an outbreak could be imminent, so the emergency instrument is understandable.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:
The Bank effectively responded to changing priorities (the declining risk perceived risk from avian influenza and the arrival of pandemic influenza in Kosovo) by restructuring the project objectives and adjusting the project design.

The Bank conducted intensive supervision, with 10 missions including mid-term review and project support missions. Missions were adequately staffed with health, veterinary and communications specialists. Support missions were effective in overcoming specific barriers to progress including disbursement and procurement challenges. Mission reports were generally comprehensive, but did not report on all new indicators added in project restructuring. Continuity in supervision was boosted by low turnover in the project team (the project team leader and animal health specialists were the same throughout, the health specialist changed only once).

Audio and video conference meetings were effective in reinvigorating the project team after implementation stalled in 2009-10.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The government provided a generally high level of commitment to the project. Before the project was approved, the government established an Avian Influenza Working Group. The government assisted in project design through this Working Group, the Office of the Prime Minister, and the Ministries of Agriculture and Health.

There were some delays in project implementation as a result of Kosovo's declaration of independence in 2008 and associated change in government and institutional reorganization, but the government remained closely engaged with the project throughout design and implementation.

Government funds were made available for compensation purposes if needed through a specific line in the budget, though these were not drawn on.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:
The Ministry of Agriculture, Forestry and Rural Development was the implementing agency. The project ICR noted that human health and veterinary health experts coordinated effectively, particularly on the awareness component. International agencies (UNICEF, WHO, FAO) implemented and financed activities in minority areas, and implemented training activities and acquisition of vaccine and antivirus.

Initial project implementation was delayed because component coordinators already had significant existing workloads, and were strained by the additional responsibilities added by the project. High turnover in project staff (including the lead project coordinator and key staff in procurement, financial management and M&E) also hampered implementation. By midterm review, significant problems had been identified in M&E (the M&E specialist had left and data was not being collected) and procurement. These problems were eventually resolved, but contributed to the need for project extensions. The project ICR reports (page 18) that the veterinary epidemiology team was poorly managed, and that this delayed sample collection and diagnosis. Delays also meant that some training activities were not completed.

There was a decline in implementing agency commitment to the project as the perceived risk of avian influenza declined. The ICR reports (page 18) that late in the project, the project team did not function effectively, meeting only once between August 2009-June 2010, and failing to undertake agreed actions.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
While the M&E system was able to track the completion of major project activities, it contained several weaknesses in design of the M&E system design. Some indicators contained multiple goals, making the indicator difficult to measure and interpret. For example, an indicator to assess the impact of the communication program was to track the proportion of the targeted population who were aware of avian influenza and who followed good behavioral practices. But the M&E system focused only on assessing awareness of avian influenza and other diseases (which has modest direct benefits) and not the behavioral change aspect (which actually effects the likelihood of disease spread).


The system was designed to be effective in tracking impacts the case where avian influenza outbreaks occurred; some indicators were not framed in a way where they would be meaningful if no outbreak occurred. For example, the framework identified as outcome indicators that the veterinary laboratory "identifies the presence of HPAI" and that the government "culls birds at the infection point". This meant that the indicators could not be measured in the absence of outbreaks. A more useful approach would be to also assess the capacity to conduct surveillance, diagnostic, and containment activities.

Most of the results focused on completion or non-completion of particular outputs; they were not set up to track the evolution of intermediate outputs (like the time needed to conduct diagnosis) over time. Virtually all indicators were qualitative.

The results framework was revised when the project was restructured and clarified some indicators, but it did little to adjust the results framework to track the revised objectives or additional activities.

The overall responsibility for M&E was with the project coordinators in the Ministry of Agriculture and Ministry of Health.

b. M&E Implementation:
At the start of the project, M&E was coordinated by one of the epidemiologists hired for the Veterinary Epidemiology Unit; in addition to his role as an epidemiologist, he collected data from line ministries and produced some reports. But over time, M&E reports were delayed or were not produced due in part to frequent staff changes. Bank supervision missions emphasized the need to focus on M&E, but reporting remained erratic.


The ICR reported (Page 15) that the newly created animal and human disease reporting systems overburdened the existing capacity in these sectors, and that reporting should have been designed to work within existing reporting structures rather than creating new systems.

Two knowledge and practice surveys were used to assess behavioral changes, but the first was conducted far enough into the project that it may not have functioned effectively as a baseline.

a. M&E Utilization:
No evidence was provided on utilization of M&E. It appears that data was generated primarily for reporting purposes rather than to influence project management.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The project was not expected to generate significant adverse environmental impacts, and so received Environment category B. Environment-related issues addressed by the project were procedures for culling and disposal of poultry, handling medical waste, and minor civil works. No other safeguards were triggered.

The ICR reported that safeguard compliance was satisfactory throughout the project's life. The project included an Environmental Management Plan which the ICR reports "was found satisfactory" (page 8). The plan was updated to cover construction of the Veterinary Laboratory construction. There was an initial waiver for the requirement to undertake an environmental assessment during preparation. The assessment was undertaken after project effectiveness. The social assessment indicated no safeguard issues.

b. Fiduciary Compliance:
The project had regular supervision by financial management and procurement specialists. Audit reports were delivered on time and were not qualified. Due to high corruption risk and low procurement capacity, the Bank pre-reviewed every procurement package for the full project period. There were some shortcomings in record-keeping and problems due to high staff turnover, but supervision reports rated financial management as moderately satisfactory.

c. Unintended Impacts (positive or negative):
Collaboration between this project and an avian influenza control project in Albania helped to strengthen technical cooperation between veterinary and epidemiological agencies.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
Achievement of pandemic preparedness objectives was modest, as was efficiency. 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Satisfactory
Moderately Satisfactory
Some weaknesses and gaps in design and poor M&E design reduced quality at entry. 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR reports a number of lessons, including:
  • A global template can enable rapid and effective project design. The Global Program on Avian Influenza template was effective in reducing project preparation time.
  • Including terms of reference and technical specifications in the operational manual can assist in rapid implementation of an emergency project. Including specifications for basic items such as protective gear and disinfectants sped up procurement of these emergency items.
  • United Nations agencies can offer technical skills that may be lacking in the Bank or in implementing agencies. They can also be effective in supporting outreach to minority ethnic enclaves where implementing agencies may have difficulty in operating.

    IEG also finds that:
  • Projects that target a single disease face challenges in remaining relevant if that disease turns out to be less serious than thought. This need not mean that single disease projects should never be designed, but that such projects need to be open to restructuring should priorities change.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR was thorough and well written. It was generally clear in explaining the logic of project implementation. There was relatively little evidence on the outcomes of the project, but this as understandable as many outcomes were difficult to observe and M&E was weak. The ICR was effective in explaining issues that arose in supervision, and at referencing supervision reports and other documents as a source of particular findings. The ICR could have been clearer on explaining the reason for the two project extensions, and on explaining whether and how revised project activities supported measures to manage other zoonoses and other infectious diseases, or whether these were largely limited to the 2009 H1N1 influenza pandemic. The ICR annex incorrectly reported in millions of US dollars cost data that was in thousands.
The ICR included the full text of the Executive Summary of the Borrower ICR, which was well written and provided useful information.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions