|1. Project Data:
ICR Review Date Posted:
|Avian Influenza Control & Human Pandemic Preparedness & Response Project
Project Costs(US $M)
Loan/Credit (US $M)
|Agriculture and Rural Development
Cofinancing (US $M)
|Japan Ministry of Finance PHRD grant
Board Approval Date
|General public administration sector (41%), Health (34%), Agricultural extension and research (20%), Other social services (5%)|
|Rural services and infrastructure (29% - P)
Other communicable diseases (29% - P)
Pollution management and environmental health (14% - S)
Health system performance (14% - S)
Participation and civic engagement (14% - S)|
||ICR Review Coordinator:
||Alain A. Barbu
|2. Project Objectives and Components:|
a. Objectives:The project development objective as expressed in the Project Appraisal Document (page 8) is to "minimize the threat posed to humans and the poultry industry in Georgia by HPAI [highly pathogenic avian influenza] and other zoonoses in domestic poultry, and to prepare for, control, and respond to influenza pandemics and other infectious disease emergencies in humans."
The objective in the Financing Agreement (Schedule 1) is to "assist the Recipient in minimizing the threat posed to humans and to the poultry industry by the HPAI infection and other zoonoses in domestic poultry industry, and preparing for, controlling, and responding to influenza pandemics, and other infectious disease emergencies in humans."
Following standard practice, the project is evaluated against the objectives in the Financing Agreement.
b. Were the project objectives/key associated outcome targets revised during implementation?
1. Animal health (appraisal estimate $USD 5.18 million, actual $2.75 million).
This component was to strengthen preparedness and response capacity for avian influenza outbreaks in poultry by:
a) Reviewing and strengthening regulatory and legal standards governing animal diseases and veterinary services, providing technical assistance to support development of an operational national avian influenza outbreak response plan, and establishing and financing a fund to compensate farmers for poultry culled to contain avian influenza outbreaks.
b) Improving disease response capacity through training of veterinarians and farmers and purchase of equipment and supplies, improving animal disease diagnostic capacity by providing training, equipment and consumable items to upgrade the capacity of border inspection posts and regional laboratories, and strengthening animal disease information systems by supporting epidemiological studies and providing GIS and database programs.
During implementation, the government decided that it would fund compensation as needed out of its own budget, rather than contributing its own or project financing into a dedicated fund that may not be needed. Regulatory, legal standard, and veterinary service reviews and national animal health program planning were dropped as they were funded by other donors.
2. Human health (appraisal estimate $USD 4.32 million, actual $4.95 million)
This component was to increase the capacity to respond to avian influenza outbreaks in humans and to enhance preparedness for an influenza pandemic by:
a) Providing technical assistance to support creation of an influenza pandemic action plan.
b) Strengthening influenza surveillance and diagnostic systems through upgrading a laboratory to Biosecurity Level 3 (BSL 3) and upgrading a regional diagnostic laboratory and providing a mobile laboratory, and by providing technical assistance and training for improving the health management information system.
c) Increasing health system response capacity through development policy guidelines on social distancing, expansion of cold storage facilities that store vaccines and other medical products, purchasing antiviral drugs, and technical assistance for medical services planning.
During implementation, it was decided that a BSL 3 laboratory was not required or economically justified and that it would be acceptable to send samples to international laboratories for retesting, and so this activity was dropped. Additional human health activities were added, including revision of clinical guidelines for influenza, development of a blood products strategy and plan, and improvements in clinical waste management.
3. Public awareness and information (appraisal estimate $USD 0.9 million, actual $0.53 million)
Improve awareness of and knowledge about avian influenza in government, private sector and civil society organizations, and in the general public, through advocacy, a public information campaign, and training of media spokespersons.
Most planned communication activities were implemented by other donors, and so were dropped from the project.
d. Comments on Project Cost, Financing, Borrower Contribution, and DatesThe project was to be financed by a US$3.5 million IDA grant, a $3.5 million IDA loan, a $1.6 million grant from the (Bank managed) Avian and Human Influenza Facility (AHIF), a $1.4 million grant from the Japan Policy and Human Resources Development Fund (PHRD), and a borrower contribution of $0.4 million. Actual financing was $2.54. million from IDA grant, $1.78 million from IDA loan, $1.54 million from the AHIF, $1.05 from the PHRD, and $1.77 from the borrower. The main reasons that the full amount of project funds were not spent were that the government made a decision not to create a compensation fund (and no outbreaks occurred during the project and so there was no need for compensation payments), that the BSL 3 laboratory was seen as unnecessary and was dropped, and that some activities were financed by other donors and so project funds were not needed.
The project was extended twice, for a total of 22 months, to allow time for completion of activities that had not been undertaken due to delays in procurement. The first extension was from August 31, 2009 to February 2011 and the second extended further to June 30, 2011, when the project closed.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:The project was responding to a global emergency, rather than being motivated by existing Bank strategies for Georgia. The objectives were fully consistent with the objectives of the Avian and Human Influenza Facility, a multi-donor financing mechanism established to assist developing countries in meeting financing gaps in minimizing the risks from avian influenza and a possible human influenza pandemic. The objectives were also consistent with the National Avian Influenza Preparedness Plan developed by the government and a Strategic Action plan developed for the government by the WHO.
Avian influenza poses a risk to the poultry sector and to human health. Georgia was considered to be at extreme risk due to its location on major pathways for migratory birds and its shared borders with Turkey, Russia, and Azerbaijan, all of which had outbreaks of highly pathogenic avian influenza prior to project approval. Sales of poultry products had dropped by as much as 90 percent in January 2006 following public awareness of outbreaks in Turkey and speculation about the possible spread to Georgia. An case of avian influenza in wild swans was detected in February 2006, which motivated the government's request for support from the Bank. The poultry sector of roughly 10.6 million birds contributed roughly 1.2% of GDP in 2004, and was a significant source of income for poor rural households (90 % of poultry were owned by households). Thus, avian influenza posed a significant threat, and project objectives were relevant at appraisal and remained relevant as of project closure, as avian influenza continued to pose a risk.
The appraisal document notes that institutional arrangements for disease control were well established in Georgia prior to appraisal, with a surveillance system (including sample collection and reporting) that had tracked human influenza in place since 2004. A national avian influenza plan was produced by the government prior to project approval, but more detailed operational plans were still needed. A program of upgrades for human and veterinary laboratories financed by USAID was already in progress, as were support programs from FAO and WHO. However, some gaps remained in animal surveillance, staff training, communications and awareness, the enabling environment, and other areas (appraisal document page 5).
The objective could have been more relevant if it included zoonotic diseases beyond those that affect poultry. Georgia suffered serious outbreaks of African Swine Fever (a zoonotic disease found in pigs) in 2007 which was not covered under the poultry-oriented objective, though the project was able to provide some urgent technical assistance to assess the outbreak, and a separate FAO project was able to provide the needed support.
The Relevance of Objectives are rated Substantial.
b. Relevance of Design:The project was generally well designed, and followed the broad template of the global program on avian influenza, incorporating animal health, human health, and communication components. The project did not aim to provide a comprehensive solution for avian influenza, but rather it aimed to fill gaps in the existing disease control system as part of a larger response coordinated with other donor programs.
The activities supported were relevant and would all contribute to the project objectives. Regulatory improvements would ensure that government has the necessary legal powers to control disease outbreaks. Outbreak response plans would provide a framework for coordinating response activities and planning for a potential epidemic. A compensation mechanism for birds destroyed by culling would help to encourage poultry farmers to report outbreaks, and so to allow them to be contained. Strengthening disease surveillance and diagnostic systems would allow outbreaks to be controlled and patterns to be monitored. Increasing health system response capacity would assist in mitigating a pandemic or infectious disease outbreak in humans should it occur. Awareness and information campaigns would help to encourage necessary political engagement, to improve overall government coordination of avian influenza response, and to encourage the public to take adopt appropriate behaviors that would reduce the chance of disease transmission to humans.
However, it was not clear that a BSL 3 laboratory was required for a small country such as Georgia, and this activity was dropped during implementation. The project design did not include much to improve biosecurity, and this was not covered by other donor efforts. Some equipment and supplies were not included in the project design as they were financed separately under the ongoing Primary Healthcare Project and Structural Reform Support Project.
Relevance of design is rated Substantial.
|4. Achievement of Objectives (Efficacy) :|
Little evidence is available on project outcomes and impacts. Significant support (including technical assistance, equipment, and laboratory needs) was provided by other donors, so it is difficult to attribute changes in outcomes to the project.
Objective 1: Minimize the threat posed to humans and the poultry industry by highly pathogenic avian influenza infection and other zoonoses in domestic poultry: Modest
- 30 veterinary service staff were trained on food safety and risk analysis topics.
- An operational action and contingency plan for avian influenza response was created and approved. Desktop simulation exercises and training programs were carried out as part of preparation of the plan.
- A compensation mechanism for culled birds was designed, but the government did not approve the compensation manual or establish a legal framework or policy for compensation, and the planned compensation fund was not financed. No compensation payments needed to be made as there were no avian influenza outbreaks and so no culling was required.
- A monitoring plan for epidemiological surveillance was developed including baselines, passive and active surveillance programs, and data collection and analysis.
- Some emergency supplies for outbreak control (disinfectant, protective clothing, rapid diagnostic test kits) were purchased and made available to veterinary staff, as were other equipment and 8 vehicles.
- Border inspection post upgrades were designed but were not constructed.
- Veterinary service offices were constructed or rehabilitated. Post mortem rooms were added to the central and three regional veterinary laboratories, and training and equipment was provided. A planned BSL 3 veterinary laboratory was not constructed.
- An integrated animal disease surveillance and information system was established, linking together 10 regional centers, but was not fully operational as of project closure.
- Emergency call centers were established in high risk areas, allowing farmers to report outbreaks more easily.
- The mobility and outbreak response capacity of veterinary services was increased through purchase of 30 vehicles.
- The project contributed to development of an avian influenza website and preparation and distribution of educational materials for poultry farmers and traders, veterinary services staff, and others in high risk regions. The communication materials and a public relations campaign were aimed at changing personal behaviors and so decreasing the chance of disease transmission from animals to people or between people.
- No outbreaks of avian influenza occurred in Georgia during the project, but this cannot be attributed to the project.
- Little evidence is available on the effectiveness of animal disease surveillance systems. The Bank team reports that the surveillance system is functioning effectively, but that it has not been integrated with the human health surveillance system. The ICR reports (page 10) that vehicles contributed to improving response time.
- Equipment was provided to regional laboratories, increasing the number of tests per day that could be carried out from 30-60 at regional laboratories and from 60 to 240 at the central laboratory. However, veterinary laboratories do not appear to be providing reliable diagnostic tests. In an assessment of capacity conducted in 2010, the laboratories were only able to correctly diagnose 15.5% to 51% of samples. This accuracy is not a sufficient basis on which to make decisions about culling operations.
- There was no impact on border security as planned upgrades were not constructed.
- No evidence is available on the impact of communications campaigns, as the planned survey was not conducted.
Objective 2. Prepare for, control, and respond to influenza pandemics and other infectious disease emergencies in humans: Substantial
- An operational influenza preparedness plan was prepared, and 27 hospital managers, 33 journalists and spokespersons were trained on the plan. Simulation exercises were carried out.
- 40 medical personnel were trained in use of ventilators at regional hospitals.
- Over 700 people were trained on influenza pandemic communication methods, including during the 2009-10 influenza pandemic.
- Upgrading of a human health laboratory to BSL 3 was not completed, as a decision was made that it was not necessary. Equipment that had been intended to be purchased for the central reference laboratory was not procured.
- 7 ambulances were procured and distributed to regional referral centers.
- Strategy and management planning for the central reference laboratory was upgraded. 4,500 medical staff were trained on sample collection. Sentinel sites where samples were collected from patients were established at 6 primary health centers.
- A strategy for blood and blood products was developed, including development of clinical guidelines and protocols for avian influenza and other infectious diseases.
- Some support was provided for health information systems, but most of this came through the Health Care Sector Development Project or other donors. The specific contribution of the avian influenza project were not clear.
- The project supported purchase and distribution of 14,000 doses of seasonal influenza.
- 9,000 doses of the antiviral drug Tamiflu were purchased.
- The project purchased roughly 500,000 pairs of protective gloves and 1.25 million masks, which were added to a stockpile to be used in case of infectious disease emergency.
- Intensive care units were upgraded with ventilators and other equipment across 48 health facilities.
- The M&E system did not track outcomes, so it is very difficult to assess the impact of the project.
- Sentinel sites provide weekly reports to the national center for disease control.
- The 2009-10 influenza pandemic provided some evidence that outputs contributed to the objectives. Discussions with the project team indicated that the human health surveillance system successfully detected the pandemic. Doses of Tamiflu and protective clothing were distributed to hospitals and used during the pandemic. Patients with severe cases were treated using respirators. Isolation measures and legal powers developed under the project were enforced during the pandemic for infected patients.
- The ICR reports that the fatality rate for severe cases requiring hospitalization were "2.5% and 3.75%", and that the government response was timely and exemplary.
- The ICR suggests (page 11) that the lack of panic during the 2009 pandemic indicated that the communications program was successful.
Though the project may have contributed significant economic benefits, conventional economic analysis cannot provide much evidence of economic efficiency because of the inherent uncertainties in the underlying probability of disease outbreaks and the quantitative efficacy of project activities on those probabilities.
An economic analysis conducted at appraisal estimates the economic benefits of the project by assuming that the project reduces the probability of outbreaks of avian influenza in poultry, and thus reduces economic losses from the poultry sector. The analysis assumes that the annual probability of a catastrophic outbreak is 10%, increasing by 1% per year, up to 25% per annum by 2020. While it is very difficult to estimate the probability, this probability seems high - especially given that in retrospect no outbreaks have occurred. [A catastrophic outbreak is assumed to reduce the market price of poultry products by 60%, and to destroy 40% of the capital value of the entire poultry sector.] The analysis assumes that the project will reduce the likelihood of outbreaks by 1/5. Given these assumptions, the analysis estimates that the project would yield a net present value of $149 million and an economic rate of return of 70.8%. The analysis does not include any benefits from reduced likelihood of human infection or from improved preparedness for pandemics or infectious diseases, which would be difficult to capture.
A separate economic analysis conducted at closure did not attempt to conduct an ex post analysis (which would have been difficult, as no outbreaks occurred), but rather takes forward-looking approach similar to that of the appraisal analysis. It considers on avoided direct losses to the poultry industry by reduced likelihood of outbreaks, but most benefits come from reduced transmission to humans (and so reduced cost of hospitalization, and workdays lost due to morbidity) and a reduction in mortality in humans. The analysis assumes that with or without the project, outbreaks of avian influenza would occur every 5 years, starting in 2011. The project is assumed to reduce the severity of outbreaks from 5% of poultry dying in an outbreak to 4% of poultry lost from disease and culling. It also assumes that the project would reduce the proportion of the population infected by a pandemic from 5% to 2%, and that mortality among hospitalized cases would be reduced from 3% to 1.5%. Based on these assumptions, the analysis estimates an overall economic rate of return of 50% and an NPV of $22.9 million. The difference between the appraisal and closure estimates demonstrate that the results are highly sensitive to modeling assumptions.
The project avoided inefficient use of resources by dropping some activities that were not needed. During implementation, it was decided that BSL 3 laboratories were not justified on a cost benefit basis. Existing BSL 2 labs and equipment were sufficient for achieving a basic diagnosis, and samples could be sent to a UK reference laboratory for confirmation. A BSL 3 laboratory was also being constructed by other donors, and so additional BSL 3 labs from this project could be redundant.
There is no clear evidence of inefficient use of resources. While some activities could not be carried out during the project, activities that were conducted were completed and facilities and equipment were in use. When some activities were dropped, the decision was made to cancel those parts of the IDA loan and grant rather than to try to use the funds on activities that might not be necessary to achieve project objectives. This way the funds could be preserved for other projects. However, efficiency was reduced by significant implementation delays, which required extending the emergency project by almost 2 years.
Project efficiency is rated Substantial.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated