Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Ar- Essential Public Health Functions


  
1. Project Data:   
ICR Review Date Posted:
11/11/2013   
Country:
Argentina
PROJ ID:
P090993
Appraisal
Actual
Project Name:
Ar- Essential Public Health Functions
Project Costs(US $M)
 999.7  n/a
L/C Number:
L7412
Loan/Credit (US $M)
 220.0  219.9
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  11/21/2006
 
 
Closing Date
12/31/2011 12/31/2012
Sector(s):
Health (83%), Sub-national government administration (9%), Central government administration (8%)
Theme(s):
Health system performance (33% - P) Other communicable diseases (17% - S) Indigenous peoples (17% - S) Natural disaster management (17% - S) Injuries and non-communicable diseases (16% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judith Hahn Gaubatz
Judyth L. Twigg Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:

According to the Loan Agreement (page 6) and the Project Appraisal Document (PAD, page 7), the project objectives were:

  • To increase coverage of ten prioritized Public Health Programs;*
  • To reduce the population's exposure to principal risk factors associated with collective illness; and
  • To improve the stewardship role and appropriate regulatory environment of the nation's public health system.

* Immunizations, tuberculosis, AIDS/sexually transmitted diseases (STDs), vector-borne diseases, reproductive health, life-style related diseases, safe blood, emerging infections, emergencies and disasters, and border public health.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

1. Strengthen National Health Stewardship and Capacity to Carry Out Essential Public Health Functions (Appraisal: US$161.8 million; Actual: n/a): This component aimed to improve capacity at the national level to carry out surveillance, policy-making, regulations, social communications, and research. Activities were to include: strengthening of surveillance systems and the public health laboratory network; strengthening of the regulatory environment for public health programs; strategic planning for social communication in health; promotion of traditional experiences from indigenous peoples; and technical assistance to carry out health research. This component also aimed to improve capacity at the provincial level to implement public health programs. Activities were to include: development of minimum standards and accreditation mechanisms; formulation of annual performance agreements; and establishment of a health municipalities network.

2. Strengthen Inputs Management and Performance Monitoring of Priority Public Health Programs and Preparedness for Health Emergencies (Appraisal: US$440.1 million; Actual: n/a): This component aimed to improve efficiency and equity in the provision of medical supplies, and to support a rapid response in case of a health emergency. Activities were to include: strengthening management capacity in resource allocation and logistics of distribution; implementing a monitoring system to track supplies; provision of drugs and medical supplies; and technical assistance on a national response plan for the avian influenza epidemic or other national health emergencies.

3. Implement and Expand a Package of Priority Public Health Programs at the Provincial Level (Appraisal: US$126.7 million; Actual: n/a): This component aimed to expand coverage and assure equity and quality of priority health programs at the provincial level. The ten Priority Health Programs identified in the project design were: immunizations, tuberculosis, AIDS/STDs, vector-borne diseases, reproductive health, life-style related diseases, safe blood, emerging infections, emergencies and disasters, and border public health. Activities were to include: implementation of public health programs; surveillance actions; disease control activities; health promotion; and provision of supplies and vaccines.

4. Management and Governance of the Essential Public Health Functions and Programs (Appraisal: US$72.7 million; Actual: n/a): This component was to support project management activities, including the operations of the project coordination unit in the Ministry of Health.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project cost

  • The estimated project cost was US$999.7 million. No actual project cost is reported in the ICR.

Financing
  • The Bank provided loan financing of US$220.0 million, of which US$219.9 million was disbursed.
  • US$121,267 was cancelled from the Loan in 2010 due to misprocurement.

Borrower contribution
  • The Borrower was expected to provide US$779.7 million of the project costs. No actual Borrower contribution was reported in the ICR, although the ICR states (page 16) that counterpart financing was timely and sufficient.

Dates
  • September 2011: The project was restructured in the following ways: three project indicators were modified to clarify wording on the targets and reflect new realities on the ground; loan proceeds were reallocated among categories; and the closing date was extended from December 2011 to December 2012. The closing date extension was requested to allow completion of activities.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High. In the last two decades, the country's public health system had been producing positive health trends overall, with better outcomes in infant and maternal mortality and life expectancy than the average outcomes for other Latin American countries. However, the significant economic crisis in 2001-02 highlighted inefficiencies and weaknesses in the health sector, particularly in the stewardship functions of the national Ministry of Health. The project objectives were highly relevant in this context, as well as to the country's Federal Health Plan, whose objectives were to reduce mortality and morbidity associated with collective illness. The objectives were also consistent with the Bank's Country Partnership Strategy for FY2010-2012, which includes the consolidation of improvements in health indicators (including the strengthening of health sector governance) as a primary pillar.

b. Relevance of Design:

Substantial. The project design focused on both essential stewardship functions to be carried out at the national level (i.e. monitoring and evaluation, surveillance, social communication, research) and essential programs to be delivered by the provincial governments (i.e. tuberculosis, vector-borne diseases). The latter focused on interventions that addressed collective health interests and included a performance-based element, in which shortfalls in achievements (in the ten prioritized programs) by provinces would lead to reduced allocations. An external technical auditing firm verified achievement of indicators on an annual basis. The project design also included significant capacity-building measures to strengthen stewardship responsibilities at the national level. The combination of these interventions, which addressed both stewardship and service delivery (at both the national and provincial level) were likely to lead to the intended outcomes. However, there were some shortcomings in the design of public health interventions. The efforts to encourage more systematic and uniform implementation among provinces did not sufficiently take into account variations among the different provinces. For example, some provinces had more limited human resources or more dispersed populations (thus making achievement of targets less likely).


4. Achievement of Objectives (Efficacy) :

To increase coverage of ten prioritized Public Health Programs

Achievement of this objective is rated Modest. Although there were achievements in some of the ten prioritized programs (tuberculosis, safe blood, lifestyle-related diseases), there were shortcomings in achievement, or lack of sufficient evidence and/or attribution of observed outcomes to project-financed activities (including lack of baselines), for the remaining prioritized programs.

Sector-wide outputs
The outputs reported here pertain to all ten priority public health programs listed below.

  • Installation of a computerized supply management system (SSMI) in each province to track supplies for blood safety, immunization, TB, HIV/AIDS, reproductive health, and vector-borne diseases.
  • Provision of medicines and supplies for the ten priority programs. Disbursement of these funds to the provinces were conditioned on the establishment of the SSMI in each province.
  • Expansion of the number of surveillance site nodes by 70%. Although the target was originally 90%, the government and Bank agreed to focus efforts on improving the quality of the existing nodes rather than expanding further. The ICR (page 21) reports that the timeliness and quality of surveillance improved due to the expansion of the network.
  • Construction of three public health laboratories, The planned modernization and upgrading of 12 provincial and 12 regional laboratories did not take place due to long delays in bidding processes.
  • Provision of materials for social communication and national health education campaigns.

Program-specific outputs and outcomes

Immunizations
Outputs
  • Support for improving the cold chain system for vaccines.

Outcomes
  • 21 out of the 24 provinces had at least 90% immunization coverage for BCG, DPT3, HiB, MMR, and HepB. This fell short of the target of 95% coverage in all 24 provinces. Among infants, the nationwide coverage rate for DPT3, HiB, and HepB was maintained, at 91% in 2007 and 92% in 2011; the rate for MMR was also maintained at 94% and 95%. Among children aged six, the nationwide coverage rate for DPT increased from 87% to 93%; and for MMR from 84% to 91%.
  • The number of autochthonous measles cases was maintained at zero.
  • The number of congenital rubella cases decreased to zero.

Tuberculosis
Outputs
  • Provision of DOTS treatment to TB patients. The number of treatments provided increased from 980 in 2007 to 6,638 in 2011.

Outcomes
  • 97.4% of new sputum-positive TB cases were receiving DOTS treatment in 2012, surpassing the target of 85%.

AIDS/STDs
Outputs
  • Installation of case management software to improve patient care (including clinical information, lab results, and treatment data). All provincial HIV programs and hospitals were linked electronically to the national system, with 30,000 patients registered.

No outcomes reported, including on coverage of ARV treatment or prevention activities.

Vector-borne diseases
Outputs
  • Fumigation of houses to protect from Chagas disease. The number of houses fumigated annually increased from 35,444 in 2007 to 82,191 in 2011. This fell short of the annual target of 248,771 houses.
  • Establishment of certified situation rooms for the epidemiological surveillance of vector-borne diseases in 21 provinces. This achieved the target of 20 provinces.

Outcomes
  • The number of cases from a dengue epidemic decreased from 27,000 in 2009 to 1,280 in 2010 and to 322 in 2011.
  • The number of provinces certified for interruption of vector transmission of Chagas disease was five, falling short of the target of 17.

There were no outcomes reported on leishmaniasis, which is identified in the PAD as a priority vector-borne disease.

Reproductive health
Outputs
  • Supervision visits to provincial health facilities, to ensure provision of reproductive health services. The number of supervision visits conducted increased steadily from 748 in 2009, to 921 in 2012, and to 1373 in 2011. The target was 960 visits annually.

There were no outcomes reported on reproductive health.

Lifestyle-related diseases
Outputs
  • Review and updating of regulations on consumption of tobacco-based products. More than 150 municipalities established laws for enforcement of tobacco control. The project team reported additional outputs including warning labels placed on cigarette packs, training on smoking cessation for health sector staff, a hotline for information to quit smoking, and research to better define intervention areas.
  • Conducting of risk factor health surveys.
  • Dissemination of health education materials promoting nutrition, exercise, and smoking cessation.

Outcomes
  • Prevalence of smoking among people aged 18-64 decreased from 33.4% in 2005 to 30.1% in 2009, achieving the target of 31.0%. Exposure to secondary smoke decreased from 52% in 2005 to 40.4% in 2009. Although the ICR (page 12) notes that most of the project activities related to this outcome were implemented after 2009, additional information provided by the project team (findings from the 2012 Global Adult Tobacco Survey in the country) also reflected a continuing decline in tobacco use over the project period, from 27.1% in 2009 to 22.1% in 2012.
  • According to the project team, a project-funded study demonstrated that there was a reduction in incidence of hypertension among those consuming lower-salt bread.

No outcomes reported on coverage of activities related to obesity or physical activity, which are identified in the PAD as priority issues in life-style related diseases.

Safe blood supply
Outputs
  • Construction of regional hemotherapy center.
  • Reorganization of the blood bank system to improve quality. Blood banks were consolidated into 20 central banks, while the number of hospital-based blood banks was reduced from 299 to 172.

Outcomes
  • The proportion of blood donations that were voluntary increased from 6% in 2006 to 30% in 2012, achieving the target of 30%.
  • The percentage of blood donations that are infected decreased from 10.5% in 2007 to 6.5% in 2011.

Emerging infections
Outputs
  • Preparation of National Integrated Plan for Response to an Influenza Pandemic.

Outcomes
  • The number of deaths from an H1N1 epidemic decreased from 626 at the peak of the epidemic (July 2009) to zero in January 2010.

Emergencies and disasters
Outputs
  • Preparation of National Integrated Plan for Response to an Influenza Pandemic.
  • Preparation of Health Emergency Preparedness Plan.

Outcomes
  • The number of cases from a dengue epidemic decreased from 27,000 in 2009 to 1,280 in 2010 and to 322 in 2011.
  • The number of deaths from an H1N1 epidemic decreased from 626 at the peak of the epidemic (July 2009) to zero in January 2010.

Border public health

No outcomes reported on coverage of border public health activities.

To reduce the population's exposure to principal risk factors associated with collective illness

Achievement of this objective is rated Substantial, due to evidence of reduced exposure to risk factors, particularly in tobacco use.

Outputs
  • Conducting of risk factor health surveys.
  • Dissemination of health education materials promoting nutrition, exercise, and smoking cessation.
  • Review and updating of regulations on consumption of tobacco-based products. More than 150 municipalities established laws for enforcement of tobacco control. The project team reported additional outputs including warning labels placed on cigarette packs, training on smoking cessation for health sector staff, a hotline for information to quit smoking, and research to better define intervention areas.
  • Conducting of risk factor health surveys.
  • Implementation of program to reduce salt in bread. 7,000 bakeries are enrolled in the program.

Outcomes
  • Prevalence of smoking among people aged 18-64 decreased from 33.4% in 2005 to 30.1% in 2009, achieving the target of 31.0%. Exposure to secondary smoke decreased from 52% in 2005 to 40.4% in 2009. Although the ICR (page 12) notes that most of the project activities related to this outcome were implemented after 2009, additional information provided by the project team (findings from the 2012 Global Adult Tobacco Survey in the country) also reflected a continuing decline in tobacco use over the project period (from 27.1% in 2009 to 22.1% in 2012). The survey also reported the following: 98.3% of the respondents believe that smoking tobacco causes serious illness; 91% say they know that tobacco use causes heart attacks or infarctions; 75.8% had seen or heard information about the dangers of smoking or urging smoking cessation in a media outlet; and 53.2% reported having seen health warnings on a cigarette pack in the past 30 days.
  • According to the project team, a project-funded study demonstrated that there was a reduction in incidence of hypertension among those consuming the lower-salt bread. The specific results of the study were not made available.

To improve the stewardship role and appropriate regulatory environment of the nation's public health system

Achievement of this objective is rated Substantial, due to implementation of activities related to quality assurance, surveillance, health promotion, supply distribution, and regulations.

Outputs
  • Development of accreditation instrument for provinces in delivering public health programs. However, there were significant delays in the design of this instrument, which led to incomplete implementation of this activity (all provinces underwent the first step of accreditation but did not complete the process). According to the ICR (page 23), there was very limited experience in the country, or internationally, with this type of instrument. The instrument initially prepared by the project team was "quantitative but became unwieldy," and by the mid-term review in 2010, a decision was made to drop it and develop a simpler version, which was then implemented in 2011.
  • Expansion in the number of surveillance site nodes by 70%. Although the target was originally 90%, the government and Bank agreed to focus efforts on improving the quality of existing nodes rather than expanding further. The ICR (page 21) reports that the timeliness and quality of surveillance improved due to the expansion of the network.
  • Establishment of certified situation rooms for epidemiological surveillance of vector-borne diseases in 21 provinces. This achieved the target of 20 provinces. Certification includes having three epidemiologists on contract, producing monthly reports for at least six months, and conducting a study about one of the ten priority programs per year.
  • Establishment of multidisciplinary teams with capacity to produce health promotion and education tools in 21 out of the 24 provinces.
  • Implementation of Annual Performance Agreements at both national and provincial levels. According to the ICR (page 13), the agreements were intended to promote integration of vertical programs, unification of service delivery standards, improvement in reporting (including accountability for results), and prioritization of key health programs.
  • Implementation of SSMI in all provinces, intended to promote more efficient resource allocation and delivery of supplies.
  • Introduction of case management software for HIV/AIDS patients, including electronically linking provincial programs with the national program.
  • Training for health staff on surveillance.
  • Review and updating of regulations on consumption of tobacco-based products. More than 150 municipalities established laws for enforcement of tobacco control.
  • Implementation of program to reduce salt in bread. 7,000 bakeries are enrolled in the program.
  • Development of Federal Register of Health Facilities and Federal Register of Health Professionals. These registries cover the totality of facilities and human resources in the country.
  • Conducting of studies on Burden of Disease, Behavioral Risk Factors, and Health Service Utilization and Costs.
  • Review and updating of norms and protocols for the ten priority health programs.
  • Establishment of a healthy municipalities network, which increased in membership from 182 in 2006 to 854 in 2012, surpassing the target of 750. Of those, the number that were accredited as full members increased from 71 to 485. 333 small-scale municipal health promotion projects were financed, and the network contributed to health education during the H1N1 and dengue outbreaks.

Outcomes
  • 68% of Annual Performance Agreements (APAs) submitted over the project period adequately met their commitments, according to an index combining progress on the various indicators in the agreements. The APAs contained a performance-based element, in which shortfalls in the achievement of key indicators led to 25% reductions in allocations (per indicator not achieved).
  • 22 out of the 24 provinces were assessed by an external technical auditing firm for achievement of minimum standards and norms for all ten prioritized programs.
  • 100% of reported measles and rubella cases and 93% of Hemolytic Urinary Syndrome diseases were investigated and managed according to norms. This surpassed the target of 90%.
  • All 24 provinces had functioning monitoring systems for supplies for the three key selected programs of blood safety, immunization, and TB.

5. Efficiency:


Efficiency is rated Substantial, due to evidence of efficient use of project resources. In terms of project design, the interventions included several of the "best buys" identified in the Disease Control Priorities review (immunizations, reduced tobacco use, DOTS treatment). The ICR (Annex 3) also reports that the following public health programs were assessed and that all indicated improved efficiency in the use of resources: immunizations, blood safety, TB and HIV. According to the project team, these programs combined comprised 55-60% of total project investments.

The consolidation of the blood bank network into 20 central blood banks, while significantly reducing the number of hospital-based blood banks (299 to 172), increased efficiency in blood collection and screening. Expenditures on screening supplies have been reduced, and quality assurance has improved. There has also been an increase in voluntary blood donations from 7% to 32%.

The use of the HIV monitoring system also contributed to increased efficiency in the treatment of HIV/AIDS patients (i.e. improved prescription and delivery processes for anti-retroviral drugs, follow-up with patients, timeliness and accuracy of treatment). The elapsed time for treatment authorization decreased from 15 days to 48 hours; viral load verification decreased from 30 days to immediate notification (although factors outside the project, such as new testing technology, may also have contributed).

The ICR suggests that efficiency was improved through the use of the SSMI to improve logistics of supplies and distribution (i.e. organization and optimization of inventory, reduction in the number of expired products, reductions in distribution time). However, no specific data are provided.


a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Relevance of the project objectives is High, and relevance of the project design is Substantial. Achievement of the objective to increase coverage of ten prioritized public health programs is rated Modest, due to shortcomings or lack of sufficient evidence for the majority of the prioritized programs. Achievement of the objective to reduce the population's exposure to principal risk factors associated with collective illness is rated Substantial due to evidence of impact particularly on tobacco use. Achievement of the objective to improve the stewardship role of the national public health system is rated Substantial. Efficiency is rated also Substantial due to evidence of cost-effective investment of project resources.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

A number of the institutional outputs (supply management system, surveillance network, laboratory network) are likely to continue functioning. A follow-up Bank operation (US$461.0 million) was approved by the Bank in 2010, which will sustain funding to essential supplies and medicines to deliver the priority public health programs, as well as continue use of the institutional arrangements. However, the mechanisms for management at the province level - the annual performance agreements and the accreditation tool - have yet to be firmly established. The ICR (page 23) reports interviews with health authorities in two provinces suggesting that greater ownership over the accreditation tool needs to be developed and that the instrument needs to be adapted to each province's reality.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

Project design was in large part informed by a detailed sector assessment carried out by the Bank prior to project preparation. The lengthy preparation period (pre-identification mission in March 2005 followed by the appraisal mission in October 2006) reflected the intensive preparation of institutional mechanisms and instruments (provincial responsibilities, performance agreements, costing of health activities, and mechanisms and algorithms for financial allocations). However, the risk assessment had some shortcomings, as some of the identified risks were underrated as Modest (risk of program implementation during a pre-election period, inadequate capacity among provincial authorities and health ministry staff) and materialized to a greater degree than anticipated, leading to shortcomings in project implementation at the provincial level. There were also weaknesses in the M&E framework, including the lack of comprehensiveness of indicators.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

Project implementation was slow at the start of the project period, in large part due to delays in developing the accreditation tool, which led to delays in other project activities. Project progress also stalled when national and provincial elections in 2007 brought new counterpart staff, who needed to be brought up to speed on the project. However, the dengue outbreak in 2009 triggered the project's activities for public health emergencies and provided an opportunity for the Bank and the project management unit to give the project more visibility. This led to increased buy-in from provincial governments. Continuity in the Bank team (and the presence of some technical staff in the field office) were positive factors that contributed to enhanced dialogue, consistent implementation support, and more timely resolution of problems. Due to weaknesses in financial management, the Bank team helped to develop a Governance and Anti-corruption Action Plan that led to improvements in financial management performance. However, the M&E framework, while refined to some extent during the project restructuring, still had inadequacies.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

National and provincial elections, which took place three months after project effectiveness, led to changes in administrations and the need to re-learn the project. In particular, the new Ministry of Health leadership "did not immediately perceive the strategic nature of the project and 'demoted' it from reporting to the Chief of Cabinet to a lower level. This lack of priority made it difficult to obtain buy-in from provincial governments and to obtain the necessary financing from the national programs" (ICR, page 6). However, commitment to the project increased following the dengue epidemic, during which the Bank and the project implementation team were able to demonstrate the rapidity and effectiveness of the health sector response. The ICR (page 16) reports that counterpart funds were provided in a timely and sufficient manner.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

The Ministry of Health at the national level, with the exception of the change in administrations after the election, remained largely committed to the project's objectives, including supporting the institutional arrangements developed for the provinces. The Ministry was able to implement numerous institutional changes to support the project objectives. However, health authorities at the provincial level faced significant implementation challenges. For example, the provincial programs were not accustomed to systematic reporting of outputs or being held accountable for their delivery, and had difficulties understanding their resource allocation mechanism. Provinces were slow in signing performance agreements. There were consistent weaknesses in financial management by the international financing unit in the Ministry of Heath, specifically with regards to internal controls, although some measures were taken to improve financial management performance. Procurement problems led to a shortfall in construction activities.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

There were some shortcomings in the M&E framework, as the choice of indicators was somewhat limited and did not comprehensively reflect the intended outcomes. Monitoring arrangements were clearly delineated, with specific monitoring systems to be developed as part of project implementation. Evaluative activities were also planned to assess some areas of project impact.

b. M&E Implementation:

The monitoring systems for the supply logistics and performance agreements were implemented as planned. The following modifications were made to the project results framework during project restructuring:

  • The target for the indicator on prevalence of smoking was modified to reflect a "6 percent decrease" rather than a "6 percentage-point difference." According to the ICR (page 2), the project amendment notes that the target was incorrectly worded in the PAD.
  • The target for the indicator on increased voluntary blood donations was modified to reflect an "increase to 30 percent," rather than an "increase by 50 percent."
  • The indicators specific to Chagas disease were broadened to reflect outcomes in vector-borne disease control overall.

However, other shortcomings in the M&E framework were not adequately addressed. The planned impact evaluation was not carried out, and although other evaluative studies were attempted later on in the project period, there were difficulties in obtaining baseline information.

a. M&E Utilization:

There was limited information provided in the ICR on the use of M&E to inform decision-making.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

Safeguards compliance was satisfactory. The project, classified as a Category "B" project, triggered policies on Indigenous Peoples (OP 4.10) and Environment Assessment (OP 4.01).

Indigenous Peoples Plans were prepared by all provinces with indigenous populations (except for one), following up on social evaluations to inform the Plans. Materials were provided in local languages, with activities designed to make the delivery of health activities more culturally relevant and sensitive.

Environmental action plans were prepared in response to health care waste management and laboratory construction issues. Environmental assessments of the renovated laboratories were conducted, and training was provided on health care waste management. Field visits (and review of documentation) confirmed satisfactory implementation of health care waste management actions, with the ICR (page 9) reporting "marked improvement each year."

b. Fiduciary Compliance:

Financial management: Although financial management capacity was assessed as adequate during project appraisal, audit reports throughout the project period issued qualified opinions due to significant internal control weaknesses. These included the Sources and Uses of Funds Statement, and the Designated Account. The ICR (page 9) suggests that while the qualifications were not serious, they reflected "continuing poor quality control of FM." There were also frequent delays in the submission of audit reports. A Governance and Anti-Corruption Plan was developed to address the internal control issues, and improvements were subsequently observed.

Procurement: The internal control issues discussed above also included procurement related to the management of the bidding cycle and contract management. These issues were identified in procurement reviews by the Bank team, with evidence of misprocurement in the awarding of some contracts during the project period. Consequently, US$121,267 was cancelled from the Loan.

Based on findings from the procurement review, the Bank and the government prepared a Procurement Action Plan that evolved into the Governance and Anti-corruption Action Plan. The Plan included recommendations to strengthen internal controls, which were subsequently observed through close supervision of the Plan. However, the procurement difficulties led to significant delays in the procurement of civil works and equipment, and therefore the shortfall in construction/upgrading activities.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
There were shortcomings in achievement of the objective to increase coverage of ten prioritized Public Health programs, due to shortfalls in outcomes or insufficient evidence.  
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Satisfactory
Moderately Satisfactory
There were shortcomings in the risk assessment and the M&E framework. 
Borrower Performance:
Satisfactory
Moderately Satisfactory
There were shortcomings in the performance of provincial governments, and in financial management. 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Lessons drawn from the ICR (pages 17-18), adapted by IEG:

  • Focusing on essential public health functions can lead to the improved effectiveness of public health programs. In the case of this project, the strengthened stewardship role of the central health ministry helped to improve delivery of services at the provincial level.
  • Performance-based financing in targeted priority health programs, alongside supply side improvements, can lead to quick improvements in service delivery. In the case of this project, the public health priorities were clearly marked out in performance agreements, while the project supported a medical supply management system that was used for a number of the priority health programs.

14. Assessment Recommended?

Yes
Why?
To verify outcomes, particularly for the prioritized public health programs, and also to learn lessons on the performance-based mechanism used for disbursements to provinces.

15. Comments on Quality of ICR:


The ICR is concise and overall consistent with guidelines. It provides a detailed account of project performance and outputs; however, the analysis of outcomes is incomplete in several of the ten prioritized public health programs. Lessons were well drawn from the project's implementation experience.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions