Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Ar Provincial Maternal-child Health Investment Apl 2


  
1. Project Data:   
ICR Review Date Posted:
11/11/2013   
Country:
Argentina
PROJ ID:
P095515
Appraisal
Actual
Project Name:
Ar Provincial Maternal-child Health Investment Apl 2
Project Costs(US $M)
 300  300
L/C Number:
L7409
Loan/Credit (US $M)
 300  300
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  11/02/2006
 
 
Closing Date
12/31/2011 12/31/2012
Sector(s):
Compulsory health finance (50%), Health (30%), Sub-national government administration (11%), Central government administration (9%)
Theme(s):
Population and reproductive health (40% - P) Child health (40% - P) Health system performance (20% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Pia Schneider
Judyth L. Twigg Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:


    This APL2 is the second and final phase of a $435 million APL and follows APL1, which was implemented in 9 provinces during 2004-2010 and reviewed by IEG in a Project Performance Assessment Report.

    According to the 2007 Loan Agreement (p. 5), "the objectives of the Project are: (a) to increase access by eligible uninsured mothers and children to basic health services; and (b) to strengthen the incentive framework for efficiency and focus on results between the national level and the Eligible Provinces and among Eligible Provinces and service providers by linking financing to both services actually rendered to the target population and to the achievement of the Maternal and Child Health Insurance Program (MCHIP) results as reflected by the selected ten tracers of the Trazadoras Matrix."

    Until July 2010, the APL2 covered the 15 provinces excluded from APL1. In 2010, after the APL1 closed, the 9 original provinces were added to APL2, which now covered 24 provinces; the APL2 objectives were not formally revised. But the "eligible" provinces included now all 24 provinces.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

The project had 5 components:

1. Implementation of the Maternal-Child Insurance Program (appraisal estimate: US$589 million, of which US$242.7 million from IBRD; actual: US$727.56 million, of which US$249.66 from IBRD). This component was to include: capitation payments for Plan Nacer (MCHIP) services from the National Ministry of Health to the participating provinces, equipment for basic health facilities, technical assistance and training programs for provincial ministries and health care providers, information technologies for monitoring the implementation of Plan Nacer, and technical assistance for the management of health care providers.

2. Strengthen National and Provincial Ministries of Health Stewardship Capacity (appraisal estimate: US$10.2 million from IBRD; actual: US$16.02 million from IBRD). This component was to include: improving epidemiology information and completing studies for policy formulation, and financing structural changes in the provincial ministries to set up Plan Nacer purchasing teams in all provinces.

3. Communication and Community Outreach (appraisal estimate: US$17 million from IBRD; actual: US$0.67 million from IBRD). This component was to include communication outreach and dissemination about the Plan Nacer. Communication activities were mainly eventually financed by the Government and not by the project, as originally planned. The Government's “Politica Unica de Comunication” mandates that funds under the control of sector ministries – including donor project funds – cannot be used for communications expenditures like TV, radio, media advertisements, etc. Instead, only the Government Communications Unit (which is outside of the sector ministries) is allowed to make these expenditures at the national level. Annual Plans of Communication were executed for 23 (out of 24) provinces. Various information and dissemination activities were undertaken at the national level.

4. Program Monitoring, Evaluation and Concurrent Auditing (appraisal estimate: US$14.6 million from IBRD; actual: US$16.79 million from IBRD). This component was to include strengthening capacity for monitoring and evaluation (M&E) and audits performance in the provinces, and financing an impact evaluation of the Plan Nacer.

5. Project Management and Administration (appraisal estimate: US$1.6 million from IBRD; actual:US$2.58 million from IBRD). This component was to include operational expenses of the National Health Service Purchasing Team, including travel costs to the provinces and per diems for meetings.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost: Originally, the total IBRD loan amount of $435 million was to be implemented in a three-phased APL over ten years. The APL1 was implemented from April 2004 until July 2010 in the 9 Northern Provinces and disbursed $135.8 million. In 2007, the remaining IBRD loan amount of $300 million (for the planned second and third APLs) was merged under this APL2 to expand the Plan Nacer into the remaining provinces.

Financing: The APL2 was financed by a $300 million IBRD loan that disbursed 100 percent. Originally, the provinces were to co-finance 50 percent of the capitation amount (PAD, p.17). This share was reduced to 30 percent and delayed.

Borrower Contribution: The PAD anticipated $346.3 million in borrower contribution. The borrower co-financed the program through the federal and provincial government budgets in the actual amount of $477.9 million. This was largely due to the increased costs for Component 1, and the financing changes described above for Component 3.

Dates:
First Loan Amendment on April 26, 2010: to reallocate loan proceeds to capitation expenditures, include 9 additional provinces previously covered under the APL1, expand the eligible list of services covered under the Benefit Package, and extend the closing date from 12/31/2011 to 12/31/2012. The Bank's co-financing share for capitation was changed; instead of decreasing to 40% it was to decrease to just 70%, leaving the remaining 30% to be financed by the Provinces.
Second Loan Amendment on June 26, 2012: to reallocate loan proceeds to capitation expenditures.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

High: The objectives of improving access to basic health care to uninsured mothers and children and strengthening the efficiency framework in a federal health system through performance contracts and financial incentives are highly relevant. The objectives are in line with Argentina's goal of poverty reduction and consolidation of gains in health. They overlap with the objectives of the Government's Health Sector Reform Program, approved in 2003, that aims to increase access to basic health services for the poorest mothers and children and to trigger significant changes in the relationship between the central and provincial governments and with health care providers to improve the incentive framework for efficiency and results. The objectives support the second pillar of the current Country Partnership Strategy (2010-2012), which is focused on improving social safety nets and consolidating progress in health outcomes, as well as the third pillar focusing on improved governance and efficiency in public sector service delivery. The objectives are appropriately targeted to the 40 percent of the population excluded from formal insurance coverage who report inferior access to health care and health outcomes.

b. Relevance of Design:

High: The project's design is highly relevant. It builds on the lessons learned from the APL1 which was implemented in the 9 Northern Provinces, and scales up these reforms nation-wide. The design targets poor and uninsured mothers and children who seek care in public health facilities. The results framework clearly links the objectives to the project components. The components consist of highly relevant features including financial incentives through the capitation payment system based on the number of enrollees in the Plan, fee-for-service payments for preventive care from provinces to providers, performance contracts between the central and provincial governments, external concurrent audits with results used to adjust payments to provinces and providers, a provincial purchasing team in charge of claims management, and substantial investments to ready the supply side to respond to increased demand under the Plan. The benefit package under the design was kept flexible and was adjusted to include additional medical treatment, including congenital heart disease (CHD) surgery. The results framework reported tracer indicators to assess compliance with standard treatment protocols for members and non-members. Results were used to adjust the capitation payment to the provinces, thereby setting a financial incentive to provinces to enable providers in better service provision.


4. Achievement of Objectives (Efficacy) :

The objectives include 2 sub-objectives. The outcome indicators are aggregated results for the entire catchment population of participating health facilities and do not distinguish between Plan Members and Non-members, mainly because the number of non-members became negligible over time. In addition, non-members benefited equally from supply-side interventions as there were spill-over effects.

Plan Nacer pays providers a fee-for-service amount for preventive care services provided to Plan members, setting a financial incentive to increase the number of preventive care services. This payment method also led to better reporting of service provision providers, meaning that the baseline values identified for 2007 are under-estimated.

1. To increase access by eligible uninsured mothers and children to basic health services: Modest

Outputs:
The share of eligible population enrolled in the program increased for the 15 Provinces from 0% in 2007 to 96% in 2012, surpassing the target of 80%. The enrollment rate in the 9 Northern Provinces increased from 84% in 2010 to 99% in 2012. However, enrollment increased only towards the end of the project after the Government's decision to pay a child allowance conditioned on Plan Nacer enrollment. The enrollment rate was 100% among eligible children and 63% among eligible women at the end of the project.

The benefit package was expanded to include surgery for congenital heart diseases (CHD) among infants. The number of CHD surgeries increased from 1,480 annually in 2007 to 1,832 per year in 2012. The region further clarified that the number of CHD surgeries for Plan members increased from 0 in 2009 to 825 in 2012. At the same time, the number of patients on the waiting list decreased from 809 to 324, or by 60%. Of the 1,689 CHD beneficiary households, 723 participated in a telephone survey, yielding a satisfaction rate of 91.3 percent.

Outcomes:

The percentage of women with reproductive health consultations increased from 27% nationally in 2007 to 90% in the 15 Provinces and 94% in the Northern Provinces by 2012, surpassing the 60% target.

The percentage of newborns weighing >2,500 grams increased from 47% nationally in 2007 to 90% in the 15 Provinces and 91% in the North by 2012, surpassing the 85% target. Risk factors in the mother that may contribute to low birth weight include multiple pregnancies, previous low birth-weight infants, poor nutrition, heart disease or hypertension, smoking, drug addiction, alcohol abuse, lead exposure, and insufficient prenatal care. (Source: Low Birth Weight University of Maryland Medical Center).

The percentage of newborns with an APGAR score of 6 or higher increased from 47% nationally in 2007 to 93% nationally by 2012, meeting the target of 93%.

The percentage of eligible women with a first antenatal care visit before the 20th week of pregnancy increased for the eligible population from a national average of 23% in 2007 to 65% in the 15 provinces and 69% in the Northern Provinces by 2012, almost meeting the target of 70%. The region further clarified that an antenatal care visit for Plan Nacer members is not comparable to the prenatal care visit reported by the WHO statistics, as for Plan Nacer members the full protocol needs to have been followed during the antenatal care visit, and verified on a sample basis by the external concurrent audits. Still, the antenatal care rates for eligible Plan Nacer women are considerably below the national average of 91% for all pregnant women as reported by WHO.

The percentage of eligible women with virology and molecular diagnostics testing and tetanus vaccination increased from 45% nationally in 2007 to 81% in the 15 provinces and 89% in the Northern Provinces by 2012, almost meeting the target of 90%.

The percentage of children <1 year of age with well-child visits increased from 13% nationally in 2007 to 35% in the 15 Provinces and to 53% in the 9 Northern Provinces by 2012, partially meeting the target of 50%.

The percentage of children less than 18 months old covered with measles vaccine or triple viral increased from 45% nationally for the eligible population in 2007 to 60% in the 15 provinces and 85% in the Northern Provinces by 2012, not meeting the target of 95%.

The baseline values are extremely low, raising questions about the quality of data, and the extent to which utilization levels improved due to better reporting.

2. To strengthen the incentive framework for efficiency and focus on results between the national level and the Eligible Provinces and among Eligible Provinces and service providers by linking financing to both services actually rendered to the target population and to the achievement of the MCHIP results as reflected by the selected ten tracers of the Trazadoras Matrix: High

Outputs:

Plan Nacer is institutionalized, as evidenced by the functioning National Health Service Purchasing Team and the 24 Provincial Health Purchasing Teams, and 85% of authorized providers receiving equipment according to annual performance agreements.

By 2012, all 24 provinces had satisfactory reporting from concurrent and financial audits.

23 of 24 provinces executed annual communication plans by 2012, meeting the target of 20 provinces.

The percentage of provincial performance agreements successfully implemented increased from 0% in the 15 Provinces in 2007 to 79% in all provinces in 2012, surpassing the target of 60%.

The percentage of providers contracted increased from 0% in the 15 Provinces in 2007 to 95% in all provinces in 2012, surpassing the target of 50%.

Outcomes:

The share of the eligible population enrolled in the program increased for the 15 Provinces from 0% in 2007 to 96% in 2012, surpassing the target of 80%. The enrollment rate in the 9 Northern Provinces increased from 84% in 2010 to 99% in 2012. The Government's decision to pay a child allowance conditioned on Plan Nacer enrollment substantially increased enrollment.

The percentage of tracers achieved increased from 0% in the 15% Provinces in 2007 to 94% in all provinces in 2012, surpassing the target of 70% (see results reported under the first objective).

A household survey conducted in 13 Provinces in 2012 found that 72% of interviewed beneficiaries were satisfied or highly satisfied with the program, meeting the target of 70%.

An impact evaluation of Plan Nacer conducted in 6 Northern Provinces, based on data from 2004 and 2008, found an estimated cost of $1,000 per DALY saved due to reduced neo-natal mortality among Plan members, which is less than the $5,170 GNI per capita in 2006, indicating substantial efficiency gains.


5. Efficiency:

The APL2 was implemented within 5 and a half years, including a one-year project extension due to delays in procurement and co-financing by the provinces.

The PAD (p.11) estimated that APL2 would cost $646.3 million (IBRD and borrower) and cover 1.7 million additional beneficiaries. The ICR reports fewer beneficiaries and higher costs, namely 1.3 million enrollees nationwide at a total cost of $777.9 million. The project facilitated investments in what are known to be the most-effective MCH interventions, including improved availability and funding for antenatal/postnatal care and CHD surgery for infants, which is a leading cause of birth defect-associated infant illness and death.

The economic analysis at appraisal highlighted the project financing of cost-effective interventions in basic health facilities to prevent and treat the main causes of infant and maternal mortality among the poor and uninsured. The analysis estimates a Net Present Value of US$ 714 million and a rate of return of 17%.

The ICR estimates an internal rate of return of 8.5%, taking into account benefits from averted neonatal deaths, with the estimate based on the impact evaluation conducted in two Northern Provinces in 2008/2009. However, the country's infant mortality rate remained at a similar level over the 5 years of project implementation (2006: 8.5 and 2011: 7.6), and has slowed down compared to the trend before the project started (2-percentage point drop from 2001 until 2006). However, with sustained Plan enrollment and improved tracer results for basic maternal and child care, as well as improved access to CHD surgery, it is plausible to expect a reduction in infant mortality over time.

Efficiency is rated Substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
17%
100%
ICR estimate:
Yes
8.5%
100%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project's outcome is rated Satisfactory, based on high relevance of objectives and high relevance of design, substantial efficiency, modest achievement of the access objective and high achievement of the objective to strengthen the incentive framework for efficiency, and substantial efficiency. The minor shortcomings in outcomes are related to sub-optimal performance on several access indicators.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The main risk that would be detrimental to the achievement of the project development outcomes is a failure by the central and provincial governments to finance and implement the Plan Nacer. The likelihood that this risk will occur is negligible. The Government is strongly committed to the Plan, which does not contribute any fiscal risk as it currently costs between 1% to 4% of the Provincial Health budget. The Government has already decided to scale up Plan Nacer to broaden the benefit package and include additional services, and to extend coverage to additional uninsured population groups. This scale-up is supported by a follow-on IBRD credit (P106735). However, the governance risk related to procurement is considered high due to market distortions such as price differences and limited bidders that negatively impact government spending (ISR), and may therefore threaten the sustainability of the project objectives. The overall risk is thus Moderate.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

The Bank team developed the project based on the experience and early lessons from APL1. The loan was innovative as it focused on results through setting financial incentives. However, given the ample experience with the 9 Northern provinces, not enough consideration was given to varying provincial capacity, institutional complexities, and insufficient availability of valid data within the provinces. As a result, project start-up and implementation was slow.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

Supervision was regular. The team kept the project flexible and restructured the project twice to adjust to the local context. In June 2010, a Quality Assessment of the Lending Portfolio (QALP-2) was conducted to address weaknesses in slow implementation. Supervision was rated as Moderately Unsatisfactory by QALP. Following the mid-term review in November 2011, the PDO rating was downgraded to Moderately Unsatisfactory (MU), because at the time only three of the eleven PDO indicators had been fully achieved. Performance had been lagging, in particular in the Phase II provinces. The Bank team in its ISR accurately described weaknesses in project implementation ratings. However, the Bank, together with the Government, developed an action plan only at the mid-term-review, which took place late, at the original closing date. This plan was implemented only during the last year of project, but it led to substantial performance improvement only during the last project year. Thus, most of the project achievements were realized only at the very end. There were weaknesses in Monitoring and Evaluation, including delays in the impact evaluation, and indicator values did not distinguish between Plan member and non-members as originally outlaid in the PAD.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Plan Nacer program supported under the APL1 and APL2 was developed under the leadership of the National Government and the Federal Health Council (COFESA). The national Ministry of Health in collaboration with the 24 Provinces have broad oversight. The Government led these reforms first in the 9 Northern Provinces, and then throughout the country. There was continuity in Government leadership for the Plan. The Plan's enrollment rates increased when the Government made child allowance payments conditioned on Plan Nacer enrollment.

However, the Government was slow in implementing the communication component. The region clarified that the “Politica Unica de Comunication” policy mandates that funds under the control of sector ministries at the national level – including donor project funds – cannot be used for communications expenditures. Instead, only the Government Communications Unit, which is outside of the sector ministries, is allowed to make these expenditures at the national level. As provinces felt more ownership over time, they were more willing to use their own funds for communications activities. Several provincial Governments were late in co-financing the Plan. These provincial Governments were suspended from receiving capitation payments until they co-financed their outstanding contributions, which led to project delays. After the mid-term review, the Government launched an Action Plan to improve indicator performance. However, the Plan was only implemented during the last project year, focusing in particular on the lagging Phase II provinces, and was supported by a reallocation of loan proceeds approved in July 2012. By project closure, all provinces paid their Plan Nacer contributions, which was a condition to qualify for the follow-up Sumar program.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

The APL2 was implemented through the National Health Service Purchasing team located within the national Ministry of Health. This National Purchasing team worked directly with the Provincial purchasing teams. There was no change among the provincial coordinators of the Plan Nacer. There were delays in the hiring of external concurrent audits, which together with delays in counterpart financing further delayed the disbursement of capitation funds and project implementation. Several provinces had insufficient capacity to engage in a results-oriented approach. There were difficulties in billing health services, weaknesses in information systems, and under-registration and misreporting of interventions at the provider level in several provinces, which required additional efforts in project supervision by the central and provincial level teams and an increase in meetings. For the most part, these weaknesses were fully addressed only in the last project year, based on the Action Plan developed during the mid-term review in November 2011. Fiduciary arrangements were moderately satisfactory (see Section 11). The final audited tracer results were not available when the ICR was written.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The APL2 M&E design included tracer indicators that were also used by the Provinces and the central Government to measure progress in the provision of care, and as conditions to define the 40% capitation transfer to the Provinces. The M&E design was to collect and report administrative data and indicators for Plan Members and for eligible non-members, and to conduct a baseline and follow-up impact evaluation in the Phase 2 provinces. The design also included a monitoring system for income, gender, ethnicity, and geographic characteristics of the eligible non-members (PAD, p. 40). Under the 4th Component (M&E), the APL2 was to finance an in-depth project evaluation, including the completion of the baseline for impact evaluation and project impact evaluations at mid-term and closing in participating provinces. The same impact evaluation design was to be used as in APL1 (PAD, p. 13). The evaluation was to include definitions for case and control groups, and a system to monitor the income, gender, ethnicity, and geographic characteristics of the eligible population.

b. M&E Implementation:

The M&E design was not implemented as planned. The tracer indicators were collected, audited, and used for disbursement of the tracer funds to the provinces. However, the project did not report indicators for members and non-members as originally planned, which became less relevant over time when almost 100% of the eligible population was enrolled. Data collection in the provinces was problematic. A baseline survey conducted in 2 provinces was delayed and only conducted 2 years after the project had started. The mid-term and follow-up impact evaluations have not been conducted in the Phase 2 provinces. The Government’s decision to condition child allowance benefits on Plan Nacer membership had a negative effect on the randomized promotion evaluation design used by the impact evaluation team, which was subsequently abandoned. An impact evaluation is being finalized based on administrative data collected in 6 of 9 Northern Provinces (APL1).

The ICR reports that additional studies were conducted, including on health worker motivation, patient satisfaction, synergies between the Universal Child Allowance Program and the Plan Nacer, and results linked to the Congestive Heart Disease Surgery Network.

a. M&E Utilization:

The Plan Nacer did contribute to improved routine administrative data collection in the Provinces; however, data collection and M&E utilization had shortcomings compared to the design planned at appraisal. The 10 tracer indicators were used to transfer 40% of the capitation installment from the central government to the Provinces. Independent concurrent audits were conducted of results, with penalties charged for erroneous reporting. The ICR does not report on the error rates by the provinces, and how adjustments were made. The technical audits of provider data that were used for the results-based payments were conducted with delays, which slowed down disbursement; however, towards the end of the project they were on time. The Plan enrollment data were used to pay the per capita amount. The Government decided to scale up the Plan Nacer nationwide based on routine administrative data and qualitative analysis. The impact evaluation was not ready to inform this decision. The additional studies, including on health worker motivation, user satisfaction, and the congestive heart disease surgery network, showed that satisfaction among patients and health workers increased over time. The monitoring and invoicing system for CHD surgery was weak. Payments were not made as planned, and by the time of the Mid-Term Review, the CHD account had accumulated US$40 million. The subsequent higher rate of invoicing and a change from capitation to fee-for-service payments for CHD surgery reduced the accumulated amount to US$20 million by project closure, suggesting that the Plan paid on average US$10,917 for the 1,832 CHD surgeries in 2012; no cost or efficiency analysis is provided in the ICR.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The APL2 was classified as "C" category for environmental screening purposes. The Indigenous Peoples Safeguards Policy was triggered under both the APL1 and the APL2. The APL2 used the results of an Indigenous Peoples Assessment conducted on a national level and was in compliance with safeguard requirements (PAD, p. 28).

b. Fiduciary Compliance:

Procurement: The Bank identified a high procurement risk and in May 2012 an independent procurement review was conducted per request of the Government, on the basis of which the Government developed an Action Plan to address weaknesses. Procurement delays slowed down the hiring of an external auditor. The Bank rated procurement as Moderately Satisfactory. An INT investigation did not find any wrongdoings.

Financial Management: The APL2 disbursed against goods and services expenditure categories, and capitation payments of $5 per Plan Nacer member per month, setting an incentive to Provinces to increase the number of Plan members. The capitation amount was to be 50% co-financed by the Government budget, amounting to a total of $10 per capita. The 50% co-financing by the central Government was to decrease over time, as provinces would co-finance an increasing share. The capitation transfer from the central to provincial governments was divided into two installments: 60% would be a capitation payment based on the number of Plan Members, and 40% of the central transfer was to be conditioned on achieving the targets agreed for the 10 tracer indicators. External concurrent audits were conducted to verify the results reported by the Provinces. For erroneous enrollment numbers, a penalty of 20% of the capitation amount was deducted.

All Project financial statements and audit reports (including 2010) were received by the Bank generally on time, reviewed by the financial management specialist, and found acceptable. The financial auditors qualified their opinions on Project Financial Statements because of differences between the total of capitation transfers to the provinces and actual payments for medical services made by the Unidades de Seguro de Gestion Provincial (USGP).

c. Unintended Impacts (positive or negative):
None reported.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Satisfactory
Moderately Satisfactory
There were overall some moderate shortcomings in Bank performance, including in project preparation, which led to delays and shortcomings in monitoring and evaluation. The mid-term review was relatively late, and the Action Plan that was developed was implemented only in the last year of the project. The ISRs accurately describe implementation challenges. 
Borrower Performance:
Satisfactory
Moderately Satisfactory

There were weaknesses in Government co-financing, data collection, and capacity, as well as delays in procurement that slowed down project implementation. The ISRs rate implementation performance as Moderately Satisfactory. 

Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

    The ICR (p. 18) identifies several lessons:

    Results-based financing can be implemented in a simple design with only 10 indicators. In this case, the Plan Nacer generated regular transfers from the central to the local governments and health facilities with limited administrative burden.

    Even if the results-based design is simple, key elements need to be in place. These include formal and transparent payment circuits, extra-budgetary transfers from the Provincial governments to the health facilities, and health facility autonomy.

    External concurrent audits of results are important management tools. In the Plan Nacer, audits helped to improve monitoring of the process and results, sending warnings if there were errors in reporting, and encouraging greater transparency in the use of funds.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR follows guidelines and includes findings from other analyses, including from the Quality Assessment conducted by QAG. However, the ICR is not sufficiently critical compared to the Bank team's description in the ISRs. The ICR could have been more candid about project performance. The ICR is fairly descriptive and lengthy; it lacks strong analytical sections. The ICR is rated Satisfactory but only marginally. However, the Bank team provided substantial additional information to the ICR, which was incorporated into this ICRR.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions