|1. Project Data:
ICR Review Date Posted:
|Avian Influenza & Human Pandemic Preparedness & Response Apl 2 Project
Project Costs(US $M)
Loan/Credit (US $M)
|Agriculture and Rural Development
Cofinancing (US $M)
|USAID, European Commission
Board Approval Date
|General public administration sector (45%), Health (28%), Agricultural extension and research (13%), Animal production (12%), Other social services (2%)|
|Rural services and infrastructure (29% - P)
Other communicable diseases (29% - P)
Pollution management and environmental health (14% - S)
Health system performance (14% - S)
Participation and civic engagement (14% - S)|
||ICR Review Coordinator:
||Robert Mark Lacey
|2. Project Objectives and Components:|
The objective of the project was to "minimize the threat posed to humans by highly pathogenic avian influenza infection and other zoonoses in domestic poultry and prepare for, control, and respond to an influenza pandemic and other infectious disease emergencies in humans" (Loan Agreement, Schedule 1).
The objective in the project appraisal document was substantially identical (page 8).
b. Were the project objectives/key associated outcome targets revised during implementation?
1. Animal Health (US$30.86 million at appraisal, $32.24 million final), including support for improving the regulatory and legal framework to meet OIE guidelines, improvements to disease surveillance systems including epidemiological studies and disease information systems, improvements to diagnostic capacity including the construction and equipping of two Biosecurity Level 3 (BSL 3) laboratories and equipment for two regional laboratories, training of veterinary services staff, creation of a compensation fund, support for culling and disposal of birds to control disease outbreaks, provision of protective equipment for field and laboratory workers, and support for the restructuring of the poultry sector to reduce the practice of backyard poultry through pilot programs. During implementation, funds for the poultry sector restructuring and compensation mechanism subcomponents were reallocated to the BSL 3 laboratories and a third laboratory was added.
2. Human Health (US$ 20.75 million at appraisal, $21.59 million final), including supply of protective equipment to health workers, upgrading drug storage facilities, improving public health and crisis information and communication systems, improvements for 7 regional public health laboratories, conducting simulation exercises for surveillance, and support for social distancing measures in the event of a pandemic.
3. Public Awareness and Communication Support (US$2.78 million at appraisal, $2.33 million final) including the design/implementation/evaluation of a national communication strategy designed to change behavior, coordination between animal health and human health government agencies at the national, provincial and local level, and to support the project management unit and monitoring and evaluation.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
The project was financed through a Euro 27.3 million IBRD loan (US$ 34.4 million equivalent), with $1 million cofinancing from USAID, $12.73 million of parallel financing from the European Commission, and a $7.06 million borrower contribution. 89% of the IBRD loan was disbursed, all co-financing and parallel financing was provided, and the borrower exceeded its planned contribution providing a total of $11.99 million.
The $5 million in the subcomponent for compensation was not used, since the government decided to use its own funds for compensation rather than use project funds for this purpose. There were 5 reallocations during the project, the most significant of which shifted the funds from the poultry sector restructuring and the compensation fund subcomponents towards construction of three BSL 3 laboratories.
The project was extended once by 13 months to allow for completion of construction of the vaccine storage warehouse, which took longer to construct than expected. It closed on December 31, 2011.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:
The project was responding to a global emergency and to an unfolding emergency in Turkey, rather than to existing national or Bank development strategies.
Avian influenza posed a significant threat to Turkey, and so the project objectives had significant relevance. The first known outbreak among birds in Turkey was identified in October 2005 in backyard poultry. Between January-March 2006, a further 230 outbreaks were identified across 58 of Turkey's 81 provinces. These outbreaks were controlled and contained, with restrictions on movement of poultry and culling of over 2.5 million birds. The outbreaks also led to some cases in humans - 21 samples from humans tested positive for highly pathogenic avian influenza, of which 12 cases were confirmed by the WHO. 4 deaths were identified. The outbreaks also triggered widespread panic, with a significant drop in demand for poultry meat leading to a 30% drop in poultry prices within 2 weeks of the outbreak. The poultry and egg sector suffered economic losses of roughly $0.9 million per day over October 2005-March 2006. Though these outbreaks were controlled, there remained an ongoing threat. Turkey is located on migration routes of many migratory birds, which can spread avian influenza and trigger new outbreaks. And a pandemic strain of influenza transmissible between humans could arise in Turkey or elsewhere.
The avian influenza outbreaks that motivated the project demonstrated a number of gaps in the animal health and human health sectors. Animal disease surveillance systems were not effective enough to identify outbreaks early on, and this allowed the rapid and extensive spread of the disease among backyard poultry throughout the country. Veterinary staff were not well prepared to conduct culling operations, and here was insufficient protective and other equipment. Compensation mechanisms for poultry farmers affected by culling were not well defined. Veterinary laboratories were overwhelmed by the number of samples to be tested. A pandemic preparedness plan existed, but lacked sufficient attention to inter-sectoral cooperation, plans for social distancing measures, and contingency plans for maintaining services during a pandemic. Human health laboratories lacked sufficient capacity to handle large numbers of samples, and had a high rate of inaccurate diagnoses. Given these weaknesses and the urgency of the unfolding outbreaks, the use of an emergency instrument was justified.
The ICR argues (page 6) that the project objectives could have been more relevant if they were restructured to expand to animal diseases beyond the poultry sector, as other zoonotic diseases posed a significant threat. For example, there were 64,037 cases of brucellosis in humans and 236 deaths from Crimean Congo Hemorrhagic Fever in Turkey over 2005-10 (ICR page 25) but these zoonotic diseases do not affect poultry.
Relevance of objectives is rated Substantial.
b. Relevance of Design:
The project was generally well designed and its activities were likely to support achievement of the project objectives. The project followed the broad template of the global program on avian influenza, including animal health, human health, and communications components. These included both urgent needs to improve response capacity, and longer term activities to support capacity building.
The specific design of the components was customized to meet the particular capacity gaps that had become apparent in Turkey during the response to the 2005-6 outbreaks. Improvements to animal health surveillance systems would mean that disease outbreaks could be detected earlier before the disease had spread. Improvements to veterinary laboratories would improve the accuracy of diagnostic tests and reduce the time lag between sample collection and diagnostic results (by using better technology and reducing sample transport time). Protective equipment and training would increase the capacity to safely respond to outbreaks among poultry and to treat any human cases. Increasing diagnostic capacity at human health laboratories would expand capacity to be able to test large numbers of samples should a pandemic occur. Improving the ability to introduce social distancing policies would help to mitigate a pandemic should it occur. Public awareness campaigns would attempt to encourage poultry farmers and others to adopt behaviors that would to reduce the spread of disease. Communication support would help to encourage coordination between the animal and human health sectors, which is necessary for effective management of zoonotic diseases.
The poultry sector restructuring activities were intended to reduce the risk of avian influenza outbreaks by shifting the poultry sector away from backyard farming and towards commercial farming, but the scope of the required changes was overly ambitious and were not likely to be completed in an emergency project timeframe. During implementation, analysis suggested that major restructuring was not necessary or economic, and so the subcomponent was scaled down relative to the initial plans.
The project design was focused on avian influenza, but increases in veterinary service and veterinary surveillance and laboratory capacity would enhance the capacity to manage other zoonoses in the poultry industry. Similarly, improvements in human health surveillance and diagnostic capacity and vaccine storage capacity would contribute to preparedness and response capacity for other infectious disease emergencies.
Relevance of design is rated Substantial.
|4. Achievement of Objectives (Efficacy) :|
Objective 1: Minimize the threat posed to humans by highly pathogenic avian influenza infection and other zoonoses in domestic poultry: Substantial
- Emergency outbreak response plans (for animal and human health) were updated and enhanced through field and table-top simulation exercises. The ICR reports (page 20) that these exercises helped veterinary services staff, enforcement officials and other stakeholders to understand how to react to outbreaks to ensure rapid containment and strengthened institutional coordination (ICR page 14). A command and control structure for outbreak response was established.
- With EU support, the project developed a legal framework to align Turkish animal disease legislation with EU standards. It also introduced legal endorsement of "compartmentalization" biosecurity standards which could improve the ability of large commercial poultry producers to export, though no poultry producers had been recognized as compartmentalized by the end of the project.
- Some biosecurity improvements were adopted as a result of project-supported studies, including a ban on the sale of live spent hens. This would reduce the transit of live birds and so would reduce disease spread risk.
- Surveillance programs were established and epidemiological studies were carried out to identify how the disease spread an inform control measures. A national animal disease information system was upgraded by expanding the system to cover poultry.
- Veterinary field staff were trained and equipped to carry out outbreak control exercises.
- Regional animal health laboratories were upgraded with civil works, new equipment, and consumable supplies. Staff were provided with training to use the equipment to diagnose avian influenza. Three laboratories were upgraded to BSL 3 and 5 others to BSL 2. The ICR notes that two of the BSL 3 laboratories were not yet functional as of mid 2012 (page 22), but the project team reported that all three laboratories are operational as of June 2013, though they are not used much as there have not been further outbreaks of highly pathogenic avian influenza.
- Some compensation for culling had already been implemented in Turkey prior to the project in an ad hoc manner. Under the project, mechanisms for compensating farmers for culled birds were clarified and improved. Compensation payments were made, but these were financed by the government rather than by the project.
- Studies and some pilots for poultry sector restructuring were carried out, including socio-economic analyses and policy reviews. The review concluded that large scale restructuring of the poultry sector was not economic or necessary, and so further restructuring efforts were not carried out.
- An inter-ministerial strategic communications working group developed a communication strategy and undertook awareness campaigns. Communications materials (including TV and radio spots, posters/brochures and other materials) were produced and distributed to all 81 provinces.
- The ICR notes (page 10) that the increased capacity to control avian influenza cannot be fully attributed to the project, but estimate's that the project's contribution was 85%.
- Three waves of outbreaks of avian influenza in poultry have occurred in Turkey. 202 outbreaks across much of the country were identified over December 2005-March 2006. 17 outbreaks in a cluster in south-east Turkey were identified over February-April 2007. 7 outbreaks on the Black Sea coast were identified over January-March 2008. No further outbreaks have been identified since 2008. 2.5 million birds were culled in 2005-6, compared to a few thousand in 2007-8.
- 12 cases of highly pathogenic avian influenza in humans were identified and confirmed during 2006 prior to project effectiveness, including 4 deaths. No human cases were identified during project implementation.
- Though it is not possible to determine a counterfactual, the evidence is consistent with an increase in outbreak identification and control capacity that has contained avian influenza.
- The animal health laboratories have the ability to diagnose avian influenza within 3-4 hours (as compared to an average of 4 days before the project). The three BSL-3 laboratories have been as certified and diagnostic tests have been certified as 100% accurate by the international veterinary laboratory in Weybridge, UK.
- According to the ICR (pages 11, 27) consumption of poultry recovered quickly after a drop in 2006 due to the communications campaign and control of outbreaks.
Objective 2. Prepare for, control, and respond to an influenza pandemic and other infectious disease emergencies in humans: Substantial
- Influenza pandemic plans were issued and updated, and a crisis management center for infectious disease emergencies was established.
- A surveillance system for influenza and other zoonoses in humans was established and connected to international public health databases. The project supported training of 233 epidemiological trainers, who in turn trained roughly 9,000 healthcare personnel in disease surveillance techniques. Within the 1,300 hospitals in the surveillance system, at least 80 percent of clinically diagnosed cases of influenza-like-illnesses were reported to the public health agency within 48 hours. The system also tracked cases of 13 zoonotic diseases.
- Influenza diagnostic capacity was increased at two public health laboratories, though provision of equipment, supplies, and training, increasing the diagnostic capacity from 100 to 300 tests per day.
- Guidelines and protocols were established for avian influenza and for other zoonotic diseases. 250 health personnel (95% of whom were doctors) were trained in clinical management of influenza.
- A 1.5 million dose stockpile of antivirus was procured.
- A large vaccine and drug storage warehouse was constructed, increasing the capacity to store influenza and other vaccines, and reducing logistical and storage costs relative to the pre-existing scattered facilities.
- Communication products were developed, but some were not broadcast as concern about the pandemic had faded by the time they were complete.
- An influenza pandemic occurred during 2009-10, but most human health component activities had not been completed by this time. The project had only a modest impact on the pandemic, largely through the communication program. Widespread vaccination occurred to mitigate the pandemic, but this was not part of the project. The antivirus stockpile was not used during the pandemic.
- According to the project team, the surveillance system successfully detected and tracked the pandemic, providing useful epidemiological information that could guide resource allocation and treatment.
- The results of public health laboratories are regularly reconfirmed by international reference laboratories.
- No evidence was available on changes in hygiene and sanitation behavior.
- The ICR reports (page 11) that Turkey's health services and communicable disease control are better equipped to deal with infectious disease outbreaks through improvements in surveillance, diagnostic, communication, containment, and treatment capacity.
It is very difficult to conduct a meaningful economic analysis for a project of this nature because of the lack of reliable data on the probability of disease outbreaks occurring or the quantitative efficacy of particular intervention measures.
An economic analysis at appraisal was based on reducing economic losses to the poultry sector caused by avian influenza outbreaks among poultry. The losses are caused by reduction of prices and quantities of poultry purchases, a reduction in the value of capital assets due to long term demand reductions in poultry, and direct losses from culling operations. Impacts on human health are not estimated. The analysis assumes that under the project the annual likelihood of a catastrophic avian influenza event will be reduced by up to 12.8 percentage points (20 percent annual chance without the project, 7.2 percent annual chance with the project). These assumptions are highly speculative. Under these assumptions the NPV of the project is estimated to be $415.9 million and the internal rate of return is 80.8%.
An economic analysis at closure assessed benefits in both human health and poultry sector. The analysis assumed that the project would reduce the proportion of the population infected by an outbreak from 3% to 2%, and that it would reduce the death rate of those infected from 3% to 2%, and so benefits come from a reduction in hospitalizations and deaths. It is not possible to assess whether these assumptions are plausible. For the poultry sector, the benefits come from a reduced number of poultry deaths attributable to HPAI. The analysis assumes that 3% of the poultry population would die from HPAI without the project, and compares that to the 2% of the poultry population that actually died from disease or was culled during the project period. The ICR acknowledges that these parameters are difficult to estimate because of a lack of data and a lack of consensus expert opinion (page 30, 32). Based on the assumptions, the ICR estimates a base case economic rate of return of 62%, and considers sensitivity analysis that would reduce the return to 25%.
Because of the differences in methodology, the ex ante and ex post economic estimates are not comparable. While the project may well have had significant economic benefits, the severe uncertainty surrounding the assumptions of the project benefits means that the economic calculations are not meaningful. This is due to inherent and unavoidable uncertainty, and not to deficiencies in the analysis. A purely ex post analysis also does not capture the benefits from the project in reducing the potential for damage from risks which might have occurred but did not turn out to occur during implementation.
There were some delays in implementation, but most activities were completed as of project closure. The project disbursed 86% of allocated funds, and the ICR notes (page 6) that this should not be assessed as a low disbursement since much of this was due to the decision not to draw on project funds for compensation.
The ICR notes that unit costs (per capita and per unit of poultry) were lower than for avian influenza projects in other countries in the region (page 12-13). The project took some decisions during implementation that improved efficiency: when it became apparent that poultry sector restructuring would be uneconomic, this activity was discontinued.
One source of potential project inefficiency is that two of the three BSL 3 laboratories constructed under the project were not operational as of project closure. Though the laboratories are now operational, they have seen low utilization because there has not been a need for three BSL 3 animal health laboratories (there have not been further outbreaks of highly dangerous animal diseases that need level 3 biosecurity, and operating the facilities at BSL 3 would be expensive). The original project design was for two laboratories, and this was increased to three during implementation. The ICR notes (page 13) that this decision does appear to have been sufficiently based on economic criteria, but overall this constitutes a moderate shortcoming in efficiency, as the three laboratories constituted less than 11% of total project cost.
The costly mobile incinerators also proved to be of limited use, since outbreaks were often occurring simultaneously in multiple parts of the country (ICR page 17).
Overall project efficiency is rated Substantial.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated