Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Preparedness Avian Influenza Control And Human Influenza Emergency Preparedness And Control


  
1. Project Data:   
ICR Review Date Posted:
11/20/2013   
Country:
Haiti
PROJ ID:
P111667
Appraisal
Actual
Project Name:
Preparedness Avian Influenza Control And Human Influenza Emergency Preparedness And Control
Project Costs(US $M)
 US$1.56M  US$1.27M
L/C Number:
CH419
Loan/Credit (US $M)
 US$1.56M  US$1.27M
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
   US$0.0
Cofinanciers:
Board Approval Date
  07/14/2008
 
 
Closing Date
08/18/2011 06/30/2012
Sector(s):
Public administration- Agriculture fishing and forestry (63%), Public administration- Health (20%), Agricultural extension and research (7%), Animal production (5%), Health (5%)
Theme(s):
Rural policies and institutions (75% - P) Other communicable diseases (25% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Sarina Abdysheva
Stephen Hutton Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
According to the Project Appraisal Document (PAD, p.9), the project’s development objective was “to support the Government's National Plan, which aimed to minimize the threat posed to humans by highly-pathogenic avian influenza (HPAI) infection and other zoonoses, and to prepare for, control, and respond to influenza pandemics and other infectious disease emergencies in humans”.

The objectives in the Financing Agreement (Schedule 1, page 5) are substantially identical.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project contained four components:

Component I: Animal health (appraisal estimate US$0.76M, actual US$0.66M), financed support for investigation, prevention, surveillance, and preparedness for avian influenza outbreaks through:

  • Provision of knowledge on the status of AI in Haiti;
  • Strengthening surveillance facilities for early detection and mitigation of the potential impacts of AI, predominantly border control / quarantine facilities;
  • Testing sanitary measures in the context of small farm poultry production;
  • Establishment of mechanism to compensate farmers for culled birds.

Component II: Human health (appraisal estimate US$0.21M, actual US$0.17M) financed support for avian influenza prevention and mitigation at the national level, through capacity building and epidemiologic surveillance in the human health sector, and the establishment of an outbreak response mechanism. Activities included:
  • Training of health staff and workshops on preventive and mitigation measures;
  • Simulations to assess the state of implementation and effectiveness of public health measures;
  • Information campaigns to increase the general level of knowledge of public health staff and the population at large.

Component III: Communications (appraisal estimate US$0.38M, actual US$0.34M). This was to support public awareness of avian influenza in vulnerable groups, secure political and civil society support, and to prepare animal and human health institutions for increased demand for advice, products and services in case of avian influenza outbreak. Activities included:
  • Provision of technical assistance in component design, implementation and monitoring;
  • Selection and training of spokespersons to provide timely and accurate information;
  • Communication with priority stakeholders in order to promote a coordinated and coherent multisectoral response if and when the avian influenza virus would appear;
  • Community based mobilization and media support to raise awareness in rural and peri-urban areas.

Component IV: Project administration (appraisal estimate US$0.13M, actual US$0.11M) financed support for project administration including procurement and disbursement services.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: On May 3, 2010, US$ 80,000 was reallocated from compensation payments to strengthening animal and human health activities and replacing laboratory equipment damaged in the 2010 earthquake.

Financing: The project was financed by a US$ 1.56 million IDA grant. As of project closure, US$ 1.27 million was disbursed and the remaining $0.3 million was cancelled. The project was to also expected to receive an additional $1 million in financing from the Avian and Human Influenza Facility (AHIF). However, the Government was unable to meet effectiveness conditions for this facility and the funding was cancelled.

Borrower contribution: The project documentation was unclear and inconsistent on whether a borrower contribution of US$ 0.57 million was expected. The ICR reports (p. 23) that a US$ 0.3 million Borrower contribution was made.

Dates: As an emergency project with a 3 year term, the project was originally intended to close on August 18, 2011. In May 2010, the project was extended to January 15, 2013 (Restructuring Paper, p.8) to allow time for project activities to be completed following implementation delays and the catastrophic earthquake of January 12, 2010. As the midterm review determined that progress was not satisfactory, a decision was made in July 2011 to close the project early by bringing the closure date forward to June 30, 2012, and to transfer any uncompleted activities to another upcoming Bank-financed operation, the "Relaunching Agriculture: Strengthening Agriculture Public Services II Project". The project then closed on June 30 2012.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial
At appraisal, Haiti faced a serious risk from a highly pathogenic avian influenza. In December 2007, an outbreak of the low pathogenic H5N2 sub-type of avian influenza was detected in the Dominican Republic and in May 2008, the virus was detected at four location across Haiti. It was suspected that the virus was introduced through its six ports and two main airports, trade along the border and internal poultry movements (especially of fighting cocks), or through the migration of infected birds. There were concerns that the Low Pathogenic Avian Influenza would spread across the country and possibly mutate into Highly Pathogenic Avian Influenza sub-type. The poultry sector is significant to the economy of Haiti with roughly 9 million birds, and is essential to the livelihood of traditional poultry farmers (the ICR reports on page 1 that 99.5% of 775,704 farms were raising poultry, according to the 2008-2010 Agricultural Census). Outbreaks of avian influenza among poultry would have significant economic costs, could potentially infect humans, and with low probability could mutate into a strain transmissible between humans and trigger a pandemic. Haiti was also exposed to the risk of infection from other zoonoses through imports of around 2.5 million chickens per month, including from Dominican Republic. The project objectives as written were limited to minimizing the risk to humans, but the project was clearly motivated in part by reducing the threat of avian influenza to the poultry sector. Thus, the objectives may have been clearer had they also included reducing the risk among poultry. Fortunately, there is no contradiction between these objectives, as controlling the disease in birds is part of the most effective way of preventing avian influenza cases in humans.

Prior to the project, Haiti was generally unprepared for highly pathogenic avian influenza or an influenza pandemic, but had begun to act following global concerns. The Government created a National Commission on Avian and Human Influenza under the Ministry of Health, which prepared a National Program for the Epidemiologic Vigilance of Avian Influenza. Under the Global Program for Avian Influenza, the Government then prepared a National Avian Influenza Control and Pandemic Preparedness Plan covering period 2006-2010, consistent with the global strategies by FAO/OIE and WHO, and providing the basis for the project and other donor support. The existing state programs on animal health were insufficient to meet immediate needs for emergency preparedness and control of avian influenza, not only in terms of quarantine, laboratory facilities and epidemiologic surveillance, but also in terms of communications. Human health expenditures in Haiti were and are among the lowest in Latin America, and the sector is characterized by insufficient health workers, essential drugs, and equipment. Most hospitals had insufficient isolation rooms and equipment for patients with severe respiratory diseases. The objectives remain relevant even now as outbreaks of HPAI H5N1 in poultry continue in more than 12 countries, including Mexico.

Project objectives were consistent with Bank practice of responding to emergencies, and were highly relevant to the Haiti Country Assistance Strategy for FY09-12 contributing to its Pillar 3 on "reduction of vulnerability to disasters" (CAS, p.27-28).

b. Relevance of Design:
Substantial
The project followed the broad design of the Global Program on Avian Influenza template, combining animal health, human health and communication components and would support the implementation of the Government's National Plan for avian influenza as a complement to the Avian and Human Influenza Facility financed program that was underway. The design addresses several important aspects of reducing the risk of avian influenza, including surveillance, diagnosis, quarantine, outbreak control, compensation, and pandemic preparedness. The animal health component was designed to improve the ability to detect avian influenza outbreaks in poultry by strengthening surveillance and border control, and by designing a compensation system that would encourage farmers to report disease outbreaks. Epidemiological studies and training would help to increase the capacity to control outbreaks that occurred. Reducing outbreaks among animals would reduce the likelihood of human infection, and so the animal health activities would support the project objective of minimizing the threat to humans.The human health component focused on increasing capacity of health workers to identify and treat humans infected by HPAI and to mitigate a pandemic. The communication component was designed to raise awareness and communicate risks to the public and to reduce behaviors that might lead to disease transmission from poultry to humans or between humans. The project design reflected the strategies and actions identified in the National Plan. Both animal and human health components involved implementation of WHO / PAHO recommended strategic measures for preparedness regarding potential avian influenza outbreaks, and reduce transmission to humans.The project was designed to complement a separate program carried out by FAO and additional activities to be financed by the AHIF. The cancellation of the AHIF financing for those additional activities weakened the design of the Bank project by leaving gaps in the National Plan.


4. Achievement of Objectives (Efficacy) :

The project was designed to implement parts of the National Plan while other parts were to be implemented by a program financed by the Avian and Human Influenza Facility. However, the AHIF TF program was cancelled, and so some aspects of the plan were left incomplete. Due to weak monitoring and evaluation, no evidence was available on the impact of most project outputs.

(a) Minimizing the threat posed to humans by HPAI and other zoonoses: Modest
Reducing the spread of avian influenza among birds contributes to minimizing the threat posed to humans, and so animal health activities are relevant to achievement of this objective.

Outputs:

  • Studies of migratory wild bird species were conducted through eight different areas in Haiti during 2010 - 2011;
  • 59 pilot initiatives for improved poultry management were implemented to train and provide materials to improve the animal health and productivity aspects of poultry production;
  • 5000 flyers were distributed among poultry producers and 1000 posters were distributed through local offices of Ministry of Agriculture for public information;
  • 2 quarantine posts were built at border crossing and 5 newly recruited border inspectors were trained;
  • Simulation exercises on avian influenza outbreaks in poultry were carried out to test the readiness of systems for disease reporting and control, but the results of these simulations and the level of preparedness demonstrated were not reported in the ICR;
  • Training activities on animal health and avian influenza were undertaken to train 5000 farmers, 70 veterinarians, 15 quarantine inspectors, 6 lab technicians and 40 veterinary agents. ICR does not provide details of subjects of the training;
  • Several workshops and conferences on avian influenza were conducted in Haiti.

Outcomes:
  • The animal health surveillance and diagnostic systems are functioning, as samples were collected and tested for avian influenza. During 2009 - 2010, low pathogenic avian influenza was detected in 169 of 3,131 tested samples, while during 2010-2011 low pathogenic avian influenza was detected in 16 of 3,884 tested samples. However, it is not clear to what extent this represents an improvement over the baseline, or to what extent any improvements could be attributed to the project;
  • 484 districts are adequately covered by the trained staff, exceeding the original target of 40 districts (ICR, p.24);
  • There is evidence that the communications campaign led to modest behavioral changes that would reduce disease transmission between animals. According to data from the Agricultural Census, 767 farmers had adopted at least 3 behaviors that reduced disease transmission, as compared to a baseline of 0 and a target of 2,800 (ICR, p.12);
  • The time taken to report information on poultry diseases at local level declined from 7 days to 2 days. It is plausible that this improvement is due to project activities;
  • No cases of highly pathogenic avian influenza were detected in Haiti in animals or humans. Cases of low pathogenic avian influenza were identified in 20% of the 570 districts. It is unlikely that the lack of highly pathogenic avian influenza outbreaks were due to the project. It cannot be determined whether the project reduced the spread of low pathogenic avian influenza.

(b) Prepare for, control and respond to influenza pandemics and other infectious diseases emergencies in humans: Modest
Outputs:
  • A Strategic Communication Plan was prepared and endorsed by the stakeholders;
  • A guide for training public health officials on avian flu and other severe influenza diseases was developed;
  • 12 veterinary laboratory staff were trained in AHI diagnosis by the project;
  • 320 human health agents were trained in surveillance of avian influenza.

Outcomes:
  • Half of the 50 districts targeted send weekly surveillance report on time;
  • The early warning surveillance system might have increased the ability to identify and respond to influenza outbreaks, but the ICR does not provide evidence on the effectiveness of the surveillance system. The ICR does not provide any evidence that the system was able to detect the 2009 - 10 pandemic.

5. Efficiency:

There is very little evidence on which to judge project efficiency. No economic analysis was conducted at appraisal. The ICR claims that the investments were economically worthwhile (page 26), but this not supported by evidence or analysis. The ICR conducts a simple forward-looking cost-benefit analysis for different options for controlling avian influenza outbreaks in future aimed at justifying continued investments in disease prevention (page 27-30), but this is not an analysis of the outputs actually produced under the project. Such an analysis would have been very difficult to carry out, since the quantitative impact of project-financed outputs in reducing disease risk is unclear.
Project implementation was initially slow, largely due to delayed procurement caused by weaknesses in fiduciary management. The ability of the implementing agency to undertake procurement following Bank procedures did not improve until very late in the Project. Delays in project implementation were also caused by the earthquake that struck Port au Prince on January 12, 2010 followed by cholera outbreak later in the same year, which resulted in a significant shift of priorities for the Government and development partners.

Project efficiency is rated: Modest

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of the objectives and design are rated substantial and the project was largely carried out according to its design. Project objectives remain relevant to the animal and human health priorities of the country. The achievement of the objectives (efficacy) is rated modest on the minimizing the threat posed to humans by HPAI and other zoonoses and modest on preparedness, control and response to influenza pandemics and other infectious diseases emergencies in humans. ICR states that the project wasn’t able to address most of the issues and some activities were not completed, however, it does not specify them. In addition, the ICR provides no evidence that lack of outbreaks could be attributed to the project outputs. Efficiency is rated modest due to a lack of evidence, and to serious delays in implementation.

Together these lead to an outcome rating of Moderately Unsatisfactory.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

The Government remains committed to the project’s objectives and concept. It is likely to support these areas in the future, given that it has proposed and developed follow-up project funded by the World Bank "Relaunching Agriculture: Strengthening Agriculture Public Services II Project". The national human health and animal health laboratories will need sufficient resources for their operations and maintenance. There is a risk that these resources will not be provided on a regular, sustained basis, in particular on human health. The animal health laboratory has secured financing through 2016.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
The project was prepared rapidly (moving from concept to appraisal in 3 months), with the design following the broad template of the global program on avian influenza. The project was to implement Government's National Plan for avian influenza with additional funding from the Avian and Human Influenza Facility. However, the AHIF TF program was cancelled leaving a gap in implementation of the planned activities. The ICR notes (page 18) that this was a weakness, as project was dependent on activities to be financed under the Facility grant, leaving the project with objectives that were too broad to be met with the IDA grant funding alone. Rapid preparation and a limited budget for preparation ($9,000) may have contributed to weaknesses in the M&E system and a lack of stakeholder consultation.

There were other shortcomings in project preparation: first, the Bank overestimated the capacity of the Ministry of Agriculture to manage project funds and procurement, and second, the capacity of Inter-American Institute for Cooperation in Agriculture as fiduciary agency was not adequately assessed. These issues created significant delays during implementation.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
The Bank provided implementation support and addressed implementation issues on a timely manner during 9 regular missions. Bank missions included livestock and human health specialists and an environmental safeguards specialist. The Bank was proactive in identifying and attempting to resolve procurement delays throughout implementation, including considering the possibility of transferring procurement responsibilities to the Ministry of Agriculture (from February 2009). The Bank also played an important role in stressing the importance of integrating all diseases into the Government’s prevention, surveillance and control strategy, and to prepare a medium term program for all animal health activities; which represents an important paradigm shift for disease management in the country.

The project closure date was extended, however due to poor performance in implementation it was decided to close the Project early. The ICR noted (p.19) that the Bank could have closed the project even earlier, but chose not to do so as they hoped that pressure from the Bank and the government on implementing agencies could improve performance.

The ICR argued that the Bank's decision to continue with the project implementation and not close it earlier was risky but justifiable in view of the project objectives, the ownership and commitment demonstrated by the Government, and the lack of an alternative acceptable to the Government.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Government commitment was evident during project preparation and implementation, with high-level, multi agency meetings and workshops to set the project’s scope and shape its concept (PAD, pp. 4,10). Before the project started, the Government has set up the National Steering Committee on Avian Influenza to provide oversight of the project and serve as a technical advisory board (PAD, pp. 14, 34), and had developed a National Plan on avian influenza. However, the Government failed to secure financing from the Avian and Human Influenza Facility that would contribute to implementation of the National Plan. Since the project was designed to complement the strategies and activities of this program, the shortage of funds significantly delayed the implementation process and prevented some of the activities from being completed. Finally, the lack of involvement of the Ministry of Public Health in the project contributed to a lack of cross - sectoral coordination, which was an important aim of the National Plan.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
The Ministry of Agriculture was responsible for overall project oversight and coordination of implementation, and was directly responsible for implementing the animal health component. A Project Coordination Unit (PCU) was established within the Directorate for Animal Health and responsible for coordinating technical execution of the entire project. The Ministry of Health was in charge of implementing the human health component. Communication activities were coordinated by the Ministry of Communication. The project’s financial administration and procurement, was outsourced to the Inter-American Institute for Cooperation on Agriculture (IICA), which acted as a fiduciary agent.

The project experienced significant delays in launching the project and during its implementation. Stakeholder assessment to determine capacity of implementing partners to handle fiduciary functions was not done properly during project preparation and lack of capacity of implementing partners was apparent from the start. IICA’s lack of fiduciary capacity and poor performance caused continuous implementation delays. Their ability to undertake procurement following Bank procedures did not improve until very late in the Project, despite requests from the Bank and the Government. Ultimately, IICA hired a procurement specialist in November 2011, following recommendations by the mid term review.

Shortcomings included poor oversight of Safeguards in the construction of one of the Quarantine Posts (not yet operational) and the missed opportunity to strengthen their M&E system.

There was very little participation by Ministry of Health in the Project. The Ministry relied almost exclusively on a consultant (hired by the project) to liaise with PCU and implement project activities. In addition, lack of prioritization of AHI within the national human health agenda was a key factor, as well as the reduced amount of resources allocated to human health within the project.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The results framework (PAD pp. 27-29) provided indicators for each component including indicators established for the national plan, but contained several weaknesses in design. The framework focused primarily on achievement of outputs, rather than on intermediate or final outcomes. Indicators recorded completion of activities, but did little to capture the impact of those activities. The overall responsibility for M&E was with the Ministry of Agriculture, but responsibility for data collection for each indicator was not clearly defined.

b. M&E Implementation:
M&E was coordinated by Ministry of Agriculture. All partners collected data and send it to the Ministry on a regular basis. The Ministry of Agriculture used the existing M&E system and maintained the complete set of indicators throughout implementation of the Project. As for the Ministry of Health, reporting on human health was not regular and it caused major delays in monitoring. During implementation, the results framework was not revised despite the acknowledgment of the weaknesses by the project team during mid term review in 2010.

a. M&E Utilization:
No evidence was provided on utilization of M&E. It appears that data was generated primarily for reporting purposes rather than to influence project management.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The project was not expected to generate significant adverse environmental impacts, and so received Environment Category B. Environment related issues addressed by the project were procedures for culling and disposal of poultry, handling medical waste, and minor civil works. No other safeguards were triggered.

The ICR reported that there were some safety standards required by an Environmental Management Plan that were not followed due to limited capacity, lack of infrastructure and resistance to new practices. However, no specific information was further provided. It also describes lack of adherence to proper environmental standards at construction of one of the quarantine posts.

No impact assessment was conducted.

b. Fiduciary Compliance:
The project had regular supervision by the Bank team, which included financial management specialist and procurement specialist. There were major delays in project implementation in the first year due to low capacity of the fiduciary agency, which was not apparent during preparation of the project. The agency did not comply with the Bank's procurement standards as requests were frequently incomplete, incorrect or in the wrong format. No fiduciary training had been anticipated for the Agency, but the project decided to hire a procurement specialist only towards the end of the project. This helped to expedite implementation process. Though hiring of an audit firm was delayed, regular audit reports were provided.

c. Unintended Impacts (positive or negative):
The project contributed to preparation of a policy strategy document on animal health covering the period from 2010 to 2014. Contribution was also made to development of a draft Code of Laws on Animal Health and Veterinary Public Health.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Borrower Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR identifies a number of lessons, including:
  • Close coordination and cooperation between animal health and human health are key to successful implementation of investments. Involvement of all concerned partners from the government with sound inputs from technical international agencies and vision from the donors’ community led initially to the design of a comprehensive plan, which prioritized activities, avoided overlaps and major gaps in the activities.
  • The establishment of a medium to long-term prevention and control program is required. Transition from emergency response to medium-term system strengthening was envisaged in the design of the National Plan and interventions also need to focus on systemic improvements and basic capacity building which are sustainable in the medium to long term.
  • Mitigation measures to close a financing gap should be part of appraisal. Potential sources of funding should be identified during preparation.
  • Institutional strengthening in project management is the pillar to success in fragile contexts, in particular for emergency operations. Assessment of institutional needs (personnel, training, facilities, logistics and equipment), including the fiduciary agency, needs to be thoroughly carried out during project preparation to identify the most efficient implementation arrangements, and to provide for the strengthening of project management throughout the life of the Project.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is generally concise and evidence based, and included useful information from a beneficiary survey and feedback from the Government and development partners. The ICR did not contain sufficient evidence to assess project impacts, largely because of the output-oriented nature of the results framework. The ICR could have been more clear in explaining the boundaries of the project and the relationship with planned financing from the Avian and Human Influenza Facility. There were some inconsistencies across documents in dates and figures.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions