Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Avian And Human Influenza Control And Prevention


  
1. Project Data:   
ICR Review Date Posted:
06/30/2013   
Country:
Egypt
PROJ ID:
P102807
Appraisal
Actual
Project Name:
Avian And Human Influenza Control And Prevention
Project Costs(US $M)
 7.1  3.5
L/C Number:
Loan/Credit (US $M)
 7.1  3.5
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
 0  0
Cofinanciers:
Avian and Human Influenza Facility
Board Approval Date
  04/18/2007
 
 
Closing Date
07/31/2009 06/30/2011
Sector(s):
Animal production (40%), Agricultural extension and research (40%), General public administration sector (20%)
Theme(s):
Rural services and infrastructure (67% - P) Other rural development (33% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Stephen Hutton
Roy Gilbert Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The objective of the project is to "minimize the threat posed to human and poultry in Egypt by Highly Pathogenic Avian Influenza (HPAI) infection and other zoonoses in domestic poultry, and to prepare for, control and respond to an influenza pandemic and other infectious disease emergencies in humans" (Avian and Human Influenza Facility Funding Proposal page 1, Grant Agreement Article II).

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project had four components:

a) Improved disease management and surveillance (appraisal estimate $4.9 million, actual $1.7 million).
This component was to strengthen disease control and surveillance capacity by:

i) controlling outbreaks through culling and safely disposing of carcasses, through increasing biosecurity on farms through training and community awareness, and through vaccination of poultry.
ii) Training field and quarantine staff, improving animal health information flow, financing conducting wild bird and backyard bird surveys, and providing equipment and transport for field teams.
The vaccination campaign was dropped during project restructuring in 2010, and additional activities were added to support purchase of motorcycles and communication equipment for veterinary services staff.


b) Review of regulatory and legal framework of veterinary services (appraisal estimate $0.045 million, actual $0).
This component was to conduct a review of the regulatory and legal framework of veterinary services to ensure they met World Organization for Animal Health standards, and to improve them as necessary.
The component was canceled during restructuring because it was undertaken instead by the World Organization for Animal Health and FAO.

c) Improved Cairo Laboratory Services (appraisal estimate $2.1 million, actual $1.5 million).
This component was to improve veterinary diagnostic services by upgrading the veterinary laboratory in Cairo by providing training and equipment. The component would also support diagnostic capacity increases outside of Cairo through training.
During project restructuring, activities were added to support purchase of equipment and supplies for regional laboratories outside of Cairo, and to construct a Biosecurity Level 3 (BSL 3) laboratory in Cairo.

d) Project management (appraisal estimate $0.088 million, actual $0.033 million)
This component was to support establishment of a project management unit within the Ministry of Agriculture and Land Reclamation through purchase of office supplies and equipment and staff costs, as well as supporting creation of an environmental management plan and financial audits.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
The project was restructured in 2010, when funds originally intended to finance poultry vaccine purchases were shifted to instead be used for additional support for veterinary services and a BSL 3 laboratory.

The project was financed entirely by a $7.14 million grant from the Avian and Human Influenza Facility. There was no borrower contribution. $3.6 million of the grant was unspent because the vaccination campaign was dropped and because many activities were not completed.

The project was extended four times (for a total of 23 months) at the request of the Government to allow completion of the original project activities (which were behind schedule due to startup delays and slow implementation) and to allow time for additional activities added in restructuring. A fifth extension was requested but was declined by the Bank as a result of major fiduciary concerns.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
The project was responding to a global emergency, rather than being motivated by existing Bank or national development strategies for Egypt. The objectives were fully consistent with the objectives of the Avian and Human Influenza Facility, a multi-donor financing mechanism established to assist developing countries in meeting financing gaps in minimizing the risks from avian influenza and a possible human influenza pandemic. The objectives were also consistent with the Integrated National Plan for Avian and Human Influenza prepared by the Egyptian government government in 2007.

Minimizing the threats posed to humans and poultry from avian influenza was relevant to Egypt. Outbreaks of Highly Pathogenic Avian Influenza (HPAI) were reported in many countries in 2004, raising fears of damage to poultry industries and the possibility of triggering a potentially catastrophic human influenza pandemic. HPAI outbreaks in Egypt were confirmed in February, 2006, and the virus spread rapidly through the country within 3 months. By 2007, there had been roughly 50 million poultry deaths from disease or culling, and 22 confirmed cases among humans (including 13 deaths). The initial government response of culling and vaccination had not been effective in containing the outbreaks. HPAI posed a very serious threat to the poultry industry and to human health, and Egypt also faced the possibility of a pandemic strain of influenza.

An Integrated National Plan for Avian Influenza identified a number of weaknesses in Egypt's capacity and preparedness. Donors and the Government discussed these weaknesses and created a division of labor across donors; the Bank would support animal health interventions (supervising a project funded by the Avian and Human Influenza Facility and using FAO technical assistance and policy support), while human health activities would be financed by USAID and awareness raising activities would be financed by UNICEF. On the animal health side, the main weaknesses identified were limited capacity of veterinary services to conduct surveillance or outbreak control measures, lack of resources to support vaccination, and a lack of diagnostic capacity in laboratories. Some support for human health activities were also added to the ongoing Bank financed Health Sector Reform Project.

Other zoonoses, such as Foot-and-Mouth disease, are also significant problems in Egypt, and so the inclusion of other zoonoses in the objectives was also relevant.

The HPAI objective had substantial relevance. The influenza pandemic and infectious disease preparedness pandemic objective was a relevant objective that the donor community was trying to help the government of Egypt to achieve. But given that donors had decided that human health and pandemic preparedness activities were to be carried out by other agencies and not by the Bank project, the relevance of including the pandemic preparedness objective in this project was negligible.

Overall relevance of objectives is rated Modest.

b. Relevance of Design:
Some elements of the design would be effective in supporting the project objectives. Diagnostic capacity was very limited, so support for veterinary diagnostic laboratories would increase the ability to identify outbreaks of avian influenza so that they could be controlled. Training, waste disposal, disinfectant and biosecurity improvements would assist in reducing the risk from avian influenza by controlling outbreaks and so would reduce the spread of the disease. Veterinary services were severely constrained by a lack of transport, and the project support for vehicles, training, and equipment would increase the ability to conduct surveillance and containment operations.

Given the poor results from the experiences of vaccination used by Egypt prior to preparation of the Bank project, vaccination may have been the wrong strategy to use to control avian influenza outbreaks in Egypt. The ICR notes that "FAO correctly assessed and advised that a continuation with mass vaccination of poultry was not effective at the time" (page 7). But this advice was not available until 2009; at the time of preparation in 2006-7, vaccination was the planned strategy under the Integrated National Plan, and so the project design was consistent with the contemporary view of best practice. As expert opinion changed over time and evidence indicated that vaccination would not be successful, the vaccination subcomponent was halted and then dropped.

A significant design gap was the lack of an effective mechanism for compensating farmers for birds culled by the government to control disease. Without an effective compensation mechanism, the objective of minimizing the risks from avian influenza would be unlikely to be achieved. The lack of compensation meant that there would be little incentive for farmers to report sick poultry (they would be better served to try to sell their birds immediately, which risked spreading the virus further), and without reports from farmers, containment efforts through vaccination or culling would be ineffective. There had been some compensation financed by the government for culling operations conducted before the project commenced, but the mechanism was largely ineffective (the compensation price was too low, the mechanism lacked transparency, and the funds were insufficient and so were soon exhausted). The design for a more effective compensation mechanism were produced by FAO, but no funds were allocated to compensation by the project due to lack of interest by the government.

Though the project included objectives to cover both responding to avian influenza among poultry (and other zoonoses) and preparing for an influenza pandemic among humans (and other infectious disease emergences), the project design included almost nothing designed to support achieving the second objective. However, many of the activities needed to support the pandemic preparedness objective were supported by other donors, so it was appropriate for the project design to not include such activities.

Relevance of design is rated Substantial.


4. Achievement of Objectives (Efficacy) :

Objective 1: Minimize the threat posed to human and poultry in Egypt by Highly Pathogenic Avian Influenza and other zoonoses in domestic poultry: Modest
Outputs:
The purchase and distribution of vaccination was canceled, on the advice of FAO and other technical experts. The project supported training for 500 veterinary services personnel on vaccination, culling and disinfection, sample collection and transportation, quarantine, and biosafety/biosecurity activities. 38 pickup trucks were purchased for veterinary services, and staff were supplied with field equipment and protective gear. Motorcycles for veterinary service staff added in the 2010 restructuring were not procured. Animal health information flow improvements were not completed as communications equipment were not purchased. Active surveillance programs were conducted during the initial project years, but were largely abandoned after 2009 as attention shifted to the H1N1 influenza pandemic.

Laboratory equipment, building upgrades and training were provided to Cairo and regional laboratories. A design proposal for a BSL 3 laboratory was created, but procurement of the laboratory was not completed due to indecisiveness about the design and interruptions following the 2011 revolution. The Borrower notes that a lack of expertise of construction companies is also a barrier.

Outcomes:
Avian influenza is endemic now in Egypt, so the project was unsuccessful in achieving the highest priority outcome. According to the WHO, there have been 167 confirmed cases of H5N1 avian influenza in humans as of April 2012, including 60 fatalities. The number of birds infected is unknown. The lack of a monitoring and evaluation system makes it difficult to observe the impact of the project. The status or impact of vehicles supplied is unknown. Field equipment and protective gear for culling operations were not of much use since few containment operations were conducted after the date of project effectiveness.

The ICR notes (page 11, page 19) that the project significantly improved surveillance and diagnostic capabilities of the central Cairo laboratory and 5 regional laboratories. Prior to the project, only the central laboratory had the capacity to diagnose avian influenza and some other zoonoses, but by closure all six laboratories had that capability. Though there is no systematic active surveillance program, the laboratories are operational and are regularly testing samples and performing diagnostics. However, improved diagnostic capacity cannot be effective by itself in reducing the risk of avian influenza when there is no effective system for reporting outbreaks and little ability to implement major response measures.

The ICR also notes (page 13) that outbreaks of food-and-mouth disease (a zoonotic disease) were quickly identified and isolated, and attributes this to the equipment and training provided by the project.

Objective 2: Prepare for, control and respond to an influenza pandemic and other infectious disease emergencies in humans: Negligible
Project activities were not designed to support this objective. After the 2009-10 H1N1 influenza pandemic there was some possibility of restructuring the project to purchase vaccine, but this did not occur and vaccines were supplied by other donors. The project had no impact on the influenza pandemic.

5. Efficiency:

No economic analysis was conducted either at appraisal or closure. It would have been difficulty to conduct a meaningful analysis given the uncertainties in the severity of avian influenza and the efficacy of containment efforts.


Avian influenza is now endemic in Egypt - the risk is higher than it was before the project started. There was a window of time when the disease could potentially have been contained, but delays, inaction, and ineffective implementation means that window has closed. While the project did not make the outcome worse, an opportunity to effectively contain avian influenza has been missed. Avian influenza is likely to continue to cause significant losses in the poultry industry and deaths among humans.

The project suffered severe implementation delays, with a 14 month gap between approval and full staffing of the project management unit. For an emergency response project, the lack of progress early in the project cycle was a serious source of inefficiency. Slow decision making throughout the project contributed to further delays. Though there was some progress on implementation, particularly following restructuring in 2010, the project was closed with less than 50% of the grant disbursed.

Given this, efficiency is rated Negligible.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

While avian influenza posed a serious risk to Egypt and so minimizing this risk was a relevant objective, the project also included an objective to support preparedness for an influenza pandemic among humans. This second objective was already being supported by other donors, and so was not relevant, so Relevance of Objectives is rated Modest. The project design included elements relevant to reducing the risks from avian influenza, but did not include a compensation for farmers that would be necessary in order to encourage disease reporting. Consequently, Relevance of Design is rated Substantial. While some achievements were made in improving diagnostic and surveillance capacity, the project was unsuccessful in minimizing the risk posed by avian influenza, and the disease has become endemic in Egypt. The project did not include activities to support influenza pandemic preparedness response. Thus, the efficacy of the project objectives are rated Modest and Negligible. The project did not provide clear economic benefits and suffered from severe implementation delays, particularly in the year after approval, and so efficiency is rated Negligible.

Overall, the project outcome is rated Unsatisfactory.

a. Outcome Rating: Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

The results achieved by the Central Laboratory are likely to be sustained due to high ownership among laboratory personnel and management. Sustainability of gains made in the veterinary services department are uncertain due to changes in leadership and budget limitations.

After the 2009-10 influenza pandemic, attention from the government to avian influenza and veterinary services waned, veterinary services had to struggle for resources to sustain services. Unless significant funding for disease identification and control is devoted to the sector, even the limited capacity that was established during the project is likely to diminish.

Political uncertainty in Egypt following the 2011 revolution and democratic elections have complicated some activities, making it more difficult to enforce quarantine controls, to conduct surveillance, and to conduct emergency planning simulation exercises. It has also led to frequent changes in leadership in government agencies.

The project management unit was staffed by fixed-term consultants rather than civil servants, and so the unit was disestablished at the end of the project and was not mainstreamed into existing institutions.


The Risk to Development Outcome is rated High.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
The project suffered from significant weaknesses from the beginning:

  • The project design did not include a compensation mechanism due to lack of interest by the government, which jeopardized the ability to detect disease outbreaks through self-reporting by farmers. The global program on avian influenza template clearly indicated that best practice for controlling avian influenza would include the presence of a compensation mechanism.
  • There was a disconnect between the project objectives and design: the design only included activities to cover half of the project objectives. No activities were included to address the pandemic preparedness objective.
  • The initial project implementation period was two years, which was unlikely to be sufficient given the complicated project design, which included significant capacity building, training, vaccination, and surveillance improvements. As it turned out, the project was extended four times for a total of 23 months, due to difficulty in completing activities and project restructuring.
  • The appraisal document was brief and below the standards of most Bank operations, even emergency interventions. The assessment of risks was cursory, noting only two potential risks to achievement of objectives (procurement delays and lack of institutional capacity for project supervision), and ignoring risks on the commitment and capacity of the agencies to implement the project. No economic analysis was included. There was no consideration of poverty, gender or social development aspects.
  • There was no effective monitoring and evaluation system (see section 10).
  • The project chose to finance purchase of 38 pickup trucks, which would have only a modest improvement in the mobility of veterinary services. Using the funds to purchase a larger number of motorcycles for veterinary services agents may have been more effective in increasing mobility of veterinary services.
  • The design did not clearly identify the separate responsibilities of the project management unit and the two implementing agencies.

Quality-at-Entry Rating: Highly Unsatisfactory

b. Quality of supervision:
Despite the fact that the project was designed as an emergency operation, the project was very slow to get underway. As noted in the ICR (page 15), the Bank did little to address delays during the critical first two years of supervision of the emergency intervention. This constituted a major shortcoming in supervision. Bank supervision reporting was weak until 2009, and there are no filed aide memoires or back-to-office reports prior to this date (ICR page 15).
Supervision did not repair the disconnect between the project objectives and design, leaving the project with broad and unachievable objectives. The Bank missed numerous opportunities to revise the objectives at design, midterm review, and restructuring phases. Project supervision reports assigned misleadingly positive ratings, as they were considering performance towards intermediate output indicators and not towards the overall project objectives (ICR page 15).

At midterm review, the Bank recommended that the vaccination campaign be canceled (based on technical advice from FAO that it was unlikely to be effective), and the government agreed to this. The Bank worked with the government to find alternative uses for funds originally intended for vaccination, but little progress was made due to slow decision-making and the lack of formal restructuring request from the client. Eventually a restructuring was carried out, to reallocate funds to support further laboratory upgrades.

In 2009-10 the Bank provided repeated support and training on financial management to the project management unit, but financial management remained poor. The Bank repeatedly notified the government of weak project management and worked with the unit to improve quality, but was largely unsuccessful and so chose to work directly with the implementing agencies as much as possible. The Bank also recommended that the government revise its Avian Influenza Plan and adopt a new strategy, but this did not occur.

After the 2011 revolution and political instability, the Bank was unable to carry out further supervision.

Given the continued lack of progress on the project and the lack of ongoing client enthusiasm for addressing avian influenza after the 2009-10 H1N1 influenza pandemic, the Bank should have considered closing the project, rather than continuing to approve extensions.

Quality of Supervision Rating: Unsatisfactory

Overall Bank Performance Rating: Highly Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Initial government government commitment was high, but political support for combatting avian influenza declined after the 2009-10 H1N1 influenza pandemic . An Integrated National Plan for Avian Influenza was prepared in May 2007 before the project started. But the Government failed to implement some critical components of the plan: they did not implement a compensation mechanism for farmers, and this undermined disease reporting and surveillance efforts. Once it became clear that the vaccination strategy was not working, the donor community including the Bank urged a change in strategy, but the Government showed a "lack of responsiveness and continued with its ineffective and ad hoc approach in attempting to control and manage the disease" (ICR page 17).

The government was unclear about communicating its changing preferences for use of project funds: a change would be discussed, but no formal request for restructuring would be made. During the influenza pandemic the Government requested to reallocate US$2 million of grant funds from veterinary services towards vaccine purchase and to add the Ministry of Health as an implementing agency. The Bank began to respond to the request, but the request was dropped a few months later (as vaccines were supplied by other donors). This disrupted project implementation and postponed possible restructuring.

The project management unit was staffed by inexperienced personnel with limited prior knowledge of World Bank procedures.

Government Performance Rating: Unsatisfactory

b. Implementing Agency Performance:
The project was implemented by two agencies, the General Organization for Veterinary Services and the Central Laboratory for Quality Control of Poultry Production.

The Veterinary Services Organization was responsible for implementing the disease management and surveillance component. Agency leadership changed 8 times during the course of the project, leading to significant delays in decision-making and a lack of ownership. Failure to make timely decisions, particularly for the vaccination campaign cancelation, further slowed implementation. The agency did not complete two large procurement packages, for communications equipment and motorcycles, which could have enhanced surveillance and reporting.

The Central Laboratory was responsible for the regulatory review and laboratory improvement components. The Laboratory leadership showed technical expertise and demonstrated significant project ownership throughout implementation, and successfully implemented most laboratory upgrades.

Financial management was weak throughout the project. Concerns about weak financial management reporting and audit non-compliance were not resolved in a timely manner.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The project design did not include an effective monitoring and evaluation (M&E) system. There was no results framework. The appraisal document (Avian and Human Influenza Facility Funding Proposal for Stand Alone Activity) identified five performance indicators, but these were all output indicators, and there were no intermediate or final outcome indicators. None of the indicators had quantitative goals, and there were no targets beyond the first year of the project. The responsibility for completing M&E activities was not clearly assigned. The project management unit had only three staff, none of whom were M&E specialists, and there is no evidence of M&E capacity within the implementing agencies.

b. M&E Implementation:
There was no M&E specialist at any time during project implementation. Little reporting was done in the first two years. A results framework was not established until March 2011, which was 6 months before project closure, by which time it was too late to monitor the project.

a. M&E Utilization:
Little M&E data was produced, and there is no evidence that it was used to support project management.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The project triggered the Environment safeguard due to the potential impact of hazardous waste from culled carcasses and laboratories, and was assigned Category B. An environmental management plan was developed, and was included in the operational manual. The Central Laboratory appointed staff for the purpose of environmental monitoring and mitigation, and followed appropriate safe disposal protocols. No evidence is available on whether safeguard procedures for biosecurity or safe carcass disposal were followed during culling activities. No other safeguards were triggered.

b. Fiduciary Compliance:
Project financial management arrangements were generally weak, and were frequently rated unsatisfactory by supervision missions. The ICR reports (page 8) that the performance of the financial management specialist in the project management unit was consistently weak despite close support and training from the Bank. A few months before project closure, the project director and financial management specialist resigned without handing over unit information to their successors. Financial monitoring reports were frequently rejected by the Bank's financial management team because of their poor quality. Audit reports were consistently late and incomplete.

The ICR reports (page 9) that procurement was moderately satisfactory throughout implementation, but that the procurement cycle was relatively long due to weak procurement capacity and slow decision making within the veterinary services agency. Procurement of goods was delayed by difficulties surrounding the exemption of imported goods from local taxes and duties. Following the grant agreement, the implementing agencies expected to receive a waiver, but the Ministry of Finance was unable to issue an exemption certificate until well after midterm review in 2009, and suppliers refused to sign contracts without the exemption certificate. Procurement packages for motorcycles and communications equipment were not completed.

The ICR notes (page 18) that the project still needs to reconcile a balance of roughly $670,000. The Bank and Borrower confirm that as of June 2013 this issue has been resolved

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Unsatisfactory
Unsatisfactory
 
Risk to Development Outcome:
High
High
 
Bank Performance:
Unsatisfactory
Highly Unsatisfactory
Quality at entry was extremely weak, and supervision was effective only in the later years of the project. 
Borrower Performance:
Unsatisfactory
Unsatisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR identifies a number of lessons, including:
  • The emergency nature of an operation should not be a reason to compromise quality at entry. Poor quality at entry in many areas handicapped implementation.
  • Emergency operations should not set an unrealistic timeline (two years in this case) for implementing complex capacity building operations.
  • Long-term underfunding of veterinary service departments leads to weak agencies with little capacity to respond to animal health emergencies when they arise.
  • Tax exempt status for imported goods should be carefully clarified with the government during project appraisal, to avoid slowing implementation.
  • Clear plans and commitments need to be made for how supervision support will be budgeted for trust fund financed operations that do not cover supervision costs.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR was well written. It provided a full and frank assessment of the weaknesses in project design, supervision, implementation, financial management, procurement, and Bank and Borrower performance. While the ICR was very effective and detailed in discussing the implementation process, it was less effective in describing the impacts of the project or in connecting outputs to achievement of objectives. The ICR did not provide evidence on the effectiveness of outputs, including some apparently successful activities (laboratory upgrades). This is understandable given the data constraints imposed by the absence of an effective monitoring and evaluation framework or system.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions