Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Sustaining Social Gains For Economic Recovery


  
1. Project Data:   
ICR Review Date Posted:
09/13/2011   
Country:
El Salvador
Is this review for a Programmatic Series?
 No
First Project ID:
P118036
Appraisal
Actual
Project Name:
Sustaining Social Gains For Economic Recovery
Project Costs(US $M)
 100  100
L/C Number:
Loan/Credit (US $M)
 100  100
Sector Board:
Economic Policy
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  11/24/2010
 
 
Closing Date
12/31/2010 12/31/2010
Sector(s):
General education sector (40%), Health (30%), Other social services (20%), General transportation sector (10%)
Theme(s):
Health system performance (30%) Administrative and civil service reform (30%) Public expenditure financial management and procurement (20%) Nutrition and food security (10%) Social safety nets (10%)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Melise Jaud
Jorge Garcia-Garcia Ismail Arslan IEGPS2

2. Project Objectives and Components:

a. Objectives:


    The objectives of this Development Policy Loan for El Salvador were:
    (i) to protect fiscal space for social expenditures
    (ii) to protect income and consumption of the vulnerable population; and
    (iii) to strengthen the institutional capacity for policy formulation and implementation in the social sectors for economic recovery.

    The DPL, intended to support the Government’s Anti Crisis Plan and serve as a bridge between an on-going DPL and an upcoming programmatic DPL series. The objectives are taken from the program document, p. 21. The Loan Agreement does not list any objectives.

b. If this is a single DPL operation (not part of a series), were the project objectives/key associated outcome targets revised during implementation?
No

c. Policy Areas:
The three objectives encompassed eight policy areas and nine prior actions.

    I. Protect fiscal space for social expenditures
      • Policy area 1a: Rationalization of public transport subsidy.
        The focus in this area was to reduce the transport subsidy, a source of adverse fiscal pressures,and to create fiscal space for high-priority spending and public investment.
      • Policy area 1b: Protecting critical non-personnel recurrent expenditures in the social sectors and critical investments in the health sector.
        The focus in this area was to prevent cutbacks on critical non-salary expenditures and investments such as education supplies, medicines, laboratory materials and other equipment to insure the quality of public education and health services.

    II. Protecting income and consumption of the vulnerable population
      • Policy area 2a: Expansion of school feeding program to urban pre-schools and basic schools.
        The focus in this area was to improve students nutrition and school attendance through expanded coverage of the School Feeding program in urban pre-schools and primary schools.
      • Policy area 2b: Ensuring access to health services for the poor.
        The DPL supported the removal of copayments in public hospitals to facilitate access to basic health services for the poor and vulnerable individuals.
      • Policy area 2c: Protecting income of the poor.
        The DPL supported the implementation of the Temporary Income Support Program to support the income of the most vulnerable population in urban areas.

    III. Strengthening institutional capacity for policy formulation and implementation in the social sectors for economic recovery
      • Policy area: Strengthening institutional capacity and policy-making in: (3a) the Ministry of Education, (3b) the Ministry of Health and (3c) social protection.
        The DPL supported institutional reforms in the education, health and social protection sectors to facilitate the development of medium-term sectoral strategies and subsequent long-term social development.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
The single tranche stand-alone DPL was approved by the Board on November 24, 2009 for US$100 million. The Salvadoran Congress approved the loan on July 19, 2010. The DPL was fully disbursed on August 27, 2010 and closed on schedule on December 31, 2010.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Relevance of the objectives is rated high. The objectives of the DPL were relevant and focused on strategic areas—fiscal expenditure, education, health and employment—given the deterioration of the fiscal deficit (increase from 3.1 percent to 5.6 percent of GDP between 2008 and 2009) and the increase in poverty (the poverty rate reached 40 percent at the end of 2008 and remained relatively high at 37.8 percent in 2009) faced by El Salvador in 2008-2009. The DPL was requested to support a number of immediate measures under the Government’s Anti Crisis Plan and its objectives were consistent with the guiding principles of the new Country Partnership Strategy FY2010-2012.

b. Relevance of Design:

Relevance of the design is rated substantial. The design of the DPL was appropriate and focused on a limited number of areas, which was adequate given the short time horizon of the DPL and the limited experience of the new government. The government was strongly involved in the design of the result framework including the identification of prior actions and outcome indicators. However, the operation result framework revealed some insufficiencies. First, it was insufficiently outcome driven. Under pillar two the outcome indicators are rather output indicators that as a consequence partially relates to the objective. Second, as pointed out in the ICR the target under policy area (3c) was too ambitious and indeed was not met.


4. Achievement of Objectives (Efficacy) :

An acceptable macro-economic framework was maintained throughout the DPL. All prior actions were complied with by the time of the signing of the DPL. Progress achieved in the three main objectives was as follows:

Objective I. Protect fiscal space for social expenditures.

Efficacy is rated substantial, reflecting good progress made in reducing the public transport subsidy and preserving levels of high-priority public spending in education and health.

    • Policy area 1a: Rationalization of public transport subsidy. The associated prior action consisted in the enactment of a decree reducing the transportation subsidy. The total subsidy amount was reduced by 42.8 percent from US$84 million in 2009 to US$48 million in 2010. However the decree expired on December, 31 2010 and subsidies to buses and microbuses increased to close to pre-decree levels, thus limiting the scope and sustainability of increase in fiscal space.
    • Policy area 1b: Protecting critical non-personnel recurrent expenditures in the social sectors and critical investments in the health sector. The prior action was to maintain the levels of non-personnel recurrent spending in education and health in 2009 and 2010 at least at the levels of fiscal year 2008 and to ensure that the share of critical investment in total investment budgeted for hospitals was protected in 2009 and 2010. Although, the Congress did not approve the decree to reform the Extraordinary Budget for Social Investment (PEIS), the Draft 2010 General Budget was, allowing the target outcome to be met. The budget for education and health increased by US$ 87.1 million, representing a 24 percent increase relative to the 2008 level. This increase ensured that critical school supplies, medicines and laboratory materials were not absent.
    Objective II. Protecting income and consumption of the vulnerable population.

    Given the difficulty to collect data for outcome indicators, over the short time horizon of the DPL, chosen indicators for policy actions under pillar two are mostly output indicators. Efficacy in this area is rated substantial based on the good achievement of outputs rather than outcomes.
      • Policy area 2a: Expansion of school feeding program to urban pre-schools and basic schools. The prior action was to expand the program to cover an additional 764 public schools and an additional 452,800 more students in urban and rural areas. In 2010, the target was met; the program had benefited an additional 459,379 students in grade 1 to 9. The chosen outcome indicator relates to the objective of expanding the coverage of the program, but does only partially relate to the development objective of protecting income and consumption of the poor. Evidence of educational progress and learning, such as promotion rates, continuation rates and achievement tests was not available by the time the DPL closed, but data on attendance could have been gathered in the course of the program implementation. To address this shortcoming the government, as a medium term policy action, plans to carry out an evaluation of the benefits of the program and improve its cost-effectiveness with technical assistance from the Brazilian Government.
      • Policy area 2b: Ensuring access to health services for the poor. The prior actions consisted in the (a) enactment of a law prohibiting the collection of copayments in public hospitals and (b) provision of supplemental funds to the Ministry of Health to compensate for revenue losses. Co-payments in public hospitals were eliminated and the number of hospitals discharges increased by 11.1 percent between 2008 and 2010. The Ministry of Health supported hospitals and health units with an additional US$15.7 million in 2010. However, to assess achievements made in ensuring access to health services for the poor, the indicator should have focused on this group. In the absence of data by income-group a more general indicator was selected. Presumably, the elimination of co-payments will increase the demand for health services by the poor and as a result hospitals will likely discharge more of them.
      • Policy area 2c: Protecting income of the poor. The associated prior action was the establishment of the Temporary Income Support Program. The target of having the program operational in 11 municipalities was met. These 11 programs were financed by USAID which was not mentioned in the PAD nor the ICR. Regarding the chosen outcome indicator "the program is fully operational in 11 municipalities", it is more an output indicator. However, recent data on the number and characteristics of program participants, made available in the course of this review, suggest that the program successfully targeted the most vulnerable population, that is young people and women.
    Objective III. Strengthening institutional capacity for policy formulation and implementation in the social sectors for economic recovery.

    Efficacy in this area is rated modest reflecting the little progress made in all three sectors: education, health and social protection.
      • Policy area 3a: Strengthening institutional capacity and policy-making in the Ministry of Education. The prior action was the creation of a new Vice-Ministry for Science and Technology. During implementation the outcome indicator became partially irrelevant following a change in priority by the Vice-Ministry of Education. The Vice-Ministry decided to elaborate a new curriculum for primary school instead of upper secondary school as elaborating a curriculum for the latter was too ambitious.
      • Policy area 3b: Strengthening institutional capacity and policy-making in the Ministry of Health. The prior action was the creation of a Vice-Ministry of Health sector policy and a Vice-Ministry of Health Care Services to monitor the implementation of the health care model (MAIS). Out of the intended target of 80 poorest municipalities, 6 did not receive health care. Nonetheless, the proportion of individuals receiving health care attention in those 74 targeted municipalities was very substantial (93 percent).
      • Policy area 3c: Strengthening institutional capacity and policy-making in social protection: the prior action consisted in re-organizing the Technical Secretariat of the Presidency. Progress in implementing the Comunidades Solidarias Urbanas (CSU) program is slow and the target of the Technical Secretariat piloting the program in two urban municipalities was not met.

5. Efficiency (not applicable to DPLs):

6. Outcome:

The program achieved the results associated with protecting the fiscal space for social expenditures and protecting the income and consumption of the vulnerable population but fell short in achieving the results associated with the objective of strengthening institutional capacity in the education and social protection sectors. On achievements, it is worth noting that: (a) to protect the space for social expenditures the government reduced the transport subsidy by 43 percent (US$36 million) and increased expenditure in education and health by US$87 million, about 24 percent over the baseline value; (b) to protect the income of the vulnerable population the government expanded the school feeding program to cover 1.3 million students, it eliminated the co-payments in public hospitals for poor people, and expanded the temporary income support program in urban and rural areas. The program, though, fell short of achieving its targets on (a) reforming the curriculum for science and technology for upper secondary school and (b) piloting the Comunidades Solidarias Urbanas program in two municipalities.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The ICR identifies several risks to development outcome. Among those it is worth noting the vulnerability of the economy to changing commodity prices, the limited institutional capacity of the government in social sectors and the high level of public debt. The increase in fuel prices in 2010 led to an increase in transport subsidy spending. The rise in food prices has increased the cost the School Feeding program straining government resources. Therefore universalizing the program by the end of 2011 may not be feasible or sustainable. On the fiscal front, El Salvador’s fiscal balance improved in 2010, and projections suggest a sustainable fiscal position. The medium-term public debt position is sustainable, but remains sensitive to an economic slowdown and a lack of fiscal consolidation. All in all the risk to development outcome is rated moderate.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

The Bank selected relevant areas of intervention that covered issues that needed to be addressed as the crisis aggravated. Selectivity in the policy areas helped the government focus on key reforms and complete them. The preparation of the DPL was based on substantive analytical work (in the form of Public Expenditure Review for FY10, policy notes in health, education and the social sector in general) and benefited from strong and continued dialogue with the government. The Bank undertook five missions for the preparation of the operation in a period of 4 month. Macroeconomic and political risks were correctly identified and adequate mitigation actions put forward. However the result framework was insufficiently outcome-driven, and some indicators were output indicators. The economic and political situation in El Salvador at the time of preparation, as well as the tight time line of the DPL, made it difficult to collect data for outcome indicators. Therefore output indicators for which data could be collected over the course of the DPL were preferred.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:

The Bank carried out one technical and two supervision missions after the loan was approved by the Board. The bank maintained a dialogue with all parties in Congress to ensure its approval of the loan. During implementation the quality of supervision benefited from a continued dialogue with the government as the Bank began preparing a new programmatic series of DPLs.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The reforms conducted under this DPL were aligned with the Government’s Anti Crisis Plan and its reform program for economic recovery, suggesting strong ownership and commitment. The Ministry of Finance was the primary implementing agency, but the Ministry of Education, Ministry of Health, Vice Ministry of Transportation, and the Technical Secretariat of the President were involved in the implementation of the DPL. The Ministry of Finance provided competent coordination between all agencies.
All agencies but one carried out the actions they had to under the DPL. There was a small shortcoming in the monitoring of one outcome. The Technical Secretariat of the President reported incorrectly that the implementation of the CSU program in two municipalities was carried out when it was not. The ICR mission confirmed it was not.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The program set clear objectives. Most prior actions are clearly linked with indicators and objectives. Indicators were few (8), easily defined and measurable, with clear baseline and target values defined. However, some indicators were rather output indicators.

b. M&E Implementation:

The Ministry of Finance and other lead ministries collected and reported on progress on outcome indicators. Except for one shortcoming mentioned in Section 9, monitoring was adequately ensured.

a. M&E Utilization:

The Bank and the authorities used the data collected for the outcome indicators in this DPL to prepare the next programmatic DPL.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
N.A

b. Fiduciary Compliance:
N.A

c. Unintended Impacts (positive or negative):
N.A

d. Other:
N.A



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

There are two important lessons: First, it is critical to have the full commitment of both the Government and the Bank when preparing a rapid operation with a new administration that has limited experience. In this DPL the new government of El Salvador was able to work very closely with the World Bank team in designing the policy actions and outcome indicators, helping to build ownership of the program and ensure that the program's objectives were in line with the government's anti-crisis plan and framework for economic recovery. Second, a program that focuses on a small number of interventions with well defined and measurable indicators is more likely to succeed. It helps the government focus on key reforms, implement them and monitor progress achieved. This is all the more important as the government institutional capacity constraint is binding.


14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is clear and concise. It provides enough information to assess the outcome of the program. The choice of some indicators—such as those reporter for the school feeding program, or the elimination of co-payment, or the income support program— could have been justified better, notably in the light of data availability constraint faced. In the lessons learned section, the first and last lessons partially contradict each other. The first states that relevant analytical work is a critical input for engaging the dialogue with the Government on key reform areas while the fourth states that taking the risk to support a reform where the government commitment is strong even if the Bank’s level of prior analytical work is moderate may be useful to open the door for longer term engagement.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6DPL-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions