Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - First Programmatic Reducing Vulnerabilities For Growth Development Policy Credit


  
1. Project Data:   
ICR Review Date Posted:
03/31/2014   
Country:
Honduras
Is this review for a Programmatic Series?
 Yes
How many operations were planned for the series?
 2
How many were approved?
 1
Series ID:
S127331
First Project ID:
P127331
Appraisal
Actual
Project Name:
First Programmatic Reducing Vulnerabilities For Growth Development Policy Credit
Project Costs(US $M)
 86.0  86.0
L/C Number:
Loan/Credit (US $M)
 86.0  86.0
Sector Board:
Economic Policy
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  12/06/2011
 
 
Closing Date
11/15/2012 11/15/2012
Sector(s):
Central government administration (100%)
Theme(s):
Macroeconomic management (40%) Conflict prevention and post-conflict reconstruction (30%) Administrative and civil service reform (30%)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Aghassi Mkrtchyan
Clay Wescott Lourdes N. Pagaran IEGPS2

2. Project Objectives and Components:

a. Objectives:


    The First Programmatic Reduction Vulnerabilities For Growth Development Policy Credit's objective was to assist the Government in strengthening fiscal management and institutional mechanism and programs responsible for an integrated violence prevention strategy (Program Document, page i, 22). In addition, the Policy Matrix lists four sub-objectives: a) improve tax administration; b) reform the pension systems; c) achieve a fiscally sustainable wage bill; and d) improve citizen security (PAD, Annex 2). The Financing Agreement does not contain a statement of objectives.
    For the purpose of this ICR review, IEG uses the overall objective set out in the Program Document, and assesses relevance and efficacy in relation to the four sub-objectives listed in the policy matrix.

b. If this is a single DPL operation (not part of a series), were the project objectives/key associated outcome targets revised during implementation?
No

c. Policy Areas:

(i) Tax Administration Reform: The focus was on supporting Government’s efforts to improve tax administration through strengthening of the large taxpayers unit and approving a registry of large taxpayers. This included two prior actions and one indicative trigger for the second operation. The program aimed to increase the number of large taxpayers filling electronic declarations by at least 10 percent (outcome indicator).

    (ii) Pension Systems Reform: This aimed at reducing the actuarial deficits of the public pension systems, regularizing the pension regime for the teachers of the Honduran Community Education Program (PROHECO), and stabilizing the long-term financial situation of the Honduran Social Security Institute (IHSS). This policy area included three prior actions and three indicative triggers. There was one quantitative outcome indicator - 100 percent of PROHOCO teachers’ contributions are flowing into Teacher’s pension Institute (INPREMA).
    (iii) Civil Service Reform: The operation supported Government’s efforts to achieve a fiscally sustainable wage bill of civil servants. This included four prior actions and two indicative triggers. It was expected that the reforms will result in a reduction of the wage bill by at least 0.7 percent of GDP (outcome indicator).
    (iv) Improving Citizen Security: The focus was to prevent crimes and violence. This area included two prior actions and one indicative trigger for the second operation. The outcome indicator was the perception of insecurity in municipalities where the program is being implemented, measured by a UNDP survey. It was expected that this indicator will decline from 91 percent in 2010 to 80 percent in 2012.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

The program provided US$ 86 million to the Government of Honduras. It was appraised in October 2011 and approved by the Board on December 06, 2011. It became effective on December 15, 2011. The credit was disbursed in full and the program was closed as scheduled on November 15, 2012. The second operation planned under the series was canceled because of substantial deterioration of fiscal situation.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

IEG assessed the relevance of objectives as substantial for the following reasons:

    • The program addresses key challenges that Honduras Government is facing. Weak tax administration and resultant lower than desired tax to GDP ratio, together with imbalances in the pension system and lax control over public sector wages, threatened country’s macroeconomic and fiscal stability, while the high level of violence (one of the worst in the world) was a major drag on economic development.
    • The program development objective and policy areas stem from the 2012-2014 Country Partnership Strategy (CPS) of the World Bank, which has pillars on consolidation of macroeconomic stability and fiscal management, and improve citizen security. In addition, the program reflects policy priorities identified in the National Plan for 2010-22.

b. Relevance of Design:
IEG assessed the relevance of design as Modest based on a review of both strengths and weaknesses.

    Key design strengths were as follows:
      • The choice to organize the Bank’s support in a series of policy lending operations (as was initially planned) reflects existing uncertainties at program preparation. Related to this, the Bank’s decision to cancel the second operation was the right response to a deteriorating fiscal situation.
      • The credit was prepared in parallel with the CPS (the Program and CPS were presented to the Board jointly) that helped to strengthen the relevance of the program.
      • The causal chain, reflected in the policy matrix, generally linked objectives to outcome indicators for most policy areas, and in general the design of specific reform actions in each policy area reflected analytical foundations.
    Despite these strengths, there were weaknesses:
      • The program lacks focus on public financial management that goes beyond public wages, such as overall expenditure control and transparency. These areas are major sources of vulnerability in Honduras. The need to include them in the program to underpin policy dialogue and support specific reform actions was especially important because the previous policy lending operation left substantial unfinished reform agenda in these critical areas.
      • Although the program document identifies lack of enforcement and control by the tax administration as the main obstacle for improving tax collection, the program focuses only on creating a large taxpayers registry and strengthening electronic filing by large taxpayers without addressing underlying governance challenges in the tax administration and without following up on Government’s plans of comprehensive tax administration reforms presented in the Program Document of the previous operation (Emergency Recovery Development Policy Credit (ERDPC), page 23).
      • The causal chain in tax administration is not strong, and the choice of outcome indicator (increase in the number of large taxpayers filing electronically) does not reflect program’s key motivation. This negatively affects the overall strength of the program.
      • One of the components of the Pension Reform supported by the program (laws to reform INPREMA and the Public Employees’ Pension Institute) is a structural benchmark of IMF’s Stand-By Arrangement and Under the Standby Credit Facility (Request for Stand-By Arrangement, September 2010, IMF, page 56). This cross-conditionally weakens the operation's overall additionally.
      • The program was in general characterized by a weak monitoring and evaluation framework, such as choice of indicators, and choice of baseline values of some of the indicators. This affects the operation's overall design.


4. Achievement of Objectives (Efficacy) :

Efficacy of the overall objective -- " to assist the Government in strengthening fiscal management and institutional mechanism and programs responsible for an integrated violence prevention strategy " -- is assessed through the achievement of the following four sub-objectives in the policy matrix:

Improve Tax Administration (modest): The program supported Government’s efforts to strengthen tax administration through focusing on large taxpayers. The prior actions in this area included approval of a plan to strengthen the large taxpayer unit and approval of a registry of large taxpayers. The indicative trigger for the second operation was adoption of transfer pricing rules by the Government. According to the ICR, the outcome indicator exceeded program’s expectations, with 549 large taxpayers submitting electronic filing as of end of 2012 as opposed to the expected 369. However, this indicator does not adequately capture the actual state of affairs in tax administration. In particular, during program implementation, the tax to GDP ratio had declined from 14.8 percent (2011) to 14.6 percent (2012), which is significantly lower than 15.3 percent of GDP expected at the time of program preparation. The reforms to promote electronic filing and reduce taxpayers compliance costs that were also areas supported by the previous ERDPC had not resulted in improved tax collections over 2010-2012, due to tax exemptions, weak revenue administration, the expiration of tax measures adopted in 2010, and a Supreme Court ruling against changes to the income tax from 2011 (2012 IMF Article IV Report). Lack of substantial achievements in tax administration threatens Honduras fiscal stability, especially against the background of continuing pressures in fiscal expenditures.

Reforming the Pension Systems (substantial): The prior actions underpinning this policy area included submitting draft laws to the Parliament for reforming INPREMA’s and the Public Employees’ Pension Institute's payment systems; formalizing incorporation of PROHECO teachers to the INPREMA pension system; and increasing the maximum salary subject to contribution to IHSS. There was only one outcome indicator in this policy area, 100 percent of PROHECO’s teachers contribution are flowing to INPREMA, which was achieved as of the end of 2012. The adoption of the Laws submitted to the Parliament had not taken place as of completion of the ICR. The program document provides indicative medium term outcome indicators that, if monitored, could have helped to better assess efficacy of this objective. In spite of shortcomings in the monitoring and evaluation framework of the operation in this area, the Bank-supported reforms are expected to stabilize the net worth of public pension funds in real terms, and were assessed as steps forward by other development partners (2012 IMF Article IV).

Achieving a Fiscally Sustainable Wage Bill (modest): Building on the achievements of ERDPC in controlling the wage bill the program aimed to improve fiscal discipline through reforming pay adjustment methodologies and improving the control over teacher’s wages. The prior actions in this policy area included unifying pay adjustment methodologies for all public sector wages; establish a single database that covers 80 percent of public servants; creating payroll management and control unit; approving consolidated norms for human resource management in education sector. Although the outcome indicator – reduction in public wage bill as percent of GDP – has been achieved at face value, the reported decline took place largely in 2011 supported by the previous operation, while in 2012, during program implementation, there was almost no further decline in this indicator. This highlights problematic choice of baseline indicator in this policy area. That said, according to the ICR the government undertook many important actions that could have potentially strengthen Government's control over public wages. However, the design of the program unfortunately does not allow full assessment of the scope of the progress in this area.

Improving Citizen Security (modest): This policy area included two prior actions - adopting violence prevention strategy and creating the National Citizen Security Council. The program aimed at reducing perceptions of securities in relevant municipalities, which was supposed to be monitored through a survey. However as of 2013 this indicator was not available which does not allow assessing the efficacy of the program in this policy area. This review, however, recognizes that the Government has taken important actions toward reducing violence and improving citizen security under the program.

Overall macroeconomic performance: Under the program, Honduras' economy performed below expectations. The growth slowed down slightly, while lack of fiscal discipline led to substantial increase in the budget deficit causing fiscal emergency. As a result, the Bank canceled the second operation planned under the series. Honduras remains vulnerable to external shocks and continues to suffer from weak revenue mobilization capacity and lax fiscal expenditure control that undermine fiscal discipline. High level of violence and related security concerns remain serious impediments for development.

5. Efficiency (not applicable to DPLs):

6. Outcome:


IEG's outcome rating of moderately unsatisfactory reflects the substantial relevance of program's objectives, modest relevance of design, and modest efficacy of all sub-objectives except the efficacy of pension reforms which is assessed as substantial. The review recognizes that many actions had been taken by the Government during program preparation and implementation. At the same time, the review highlights that weaknesses in the design prevent one from fully assessing the effects of these actions in many areas. The review also highlights that despite Bank’s two consecutive operations focused on macro-fiscal sustainability and reducing vulnerabilities, the tax administration remains a serious challenge with very low and largely stagnant tax to GDP ratio, while weak fiscal management threatens Honduras macro-fiscal stability . These existing challenges in areas directly related to program's objective had led to a fiscal emergency during program implementation.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

Despite some progress on the political front, Honduras is still facing serious macroeconomic and institutional risks that may undermine the results achieved by the Bank-supported programs of the Government. Honduras is a small, open economy, exposed to external risks that could undermine fiscal sustainability and economic stability. Government's revenue mobilization capacity remains weak, keeping the tax to GDP ratio below its potential. Shortcomings in public financial management also undermine effectiveness of resource use and remain major bottlenecks for expanding donor assistance. High levels of violence add to the risk, which may be mitigated through Bank support to violence prevention through an ongoing operation.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:

The Bank team worked closely with the Government and other partners to put together a program that supports Government’s actions toward reducing vulnerability and increasing citizen security. Substantial expertise was put in place to identify key constraints hindering Honduras’s progress. The program preparation drew on analytical work, as well as on previous policy lending operations. The program, however, did not follow up on some of the unfinished policy agenda remaining from the prior operation that was relevant for program's objective, including reforms of public financial management beyond controlling the wage bill and tax reforms to address tax exemption and loopholes. The program does not provide political economy analysis regarding feasibility of these reforms, as well as analysis on the possible implications of lack of progress in these areas for economic stability, and, respectively, the risks to the Bank’s engagement. The program also suffered from a weak monitoring and evaluation framework (discussed in section 10).

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

Supervision was carried out through three supervisory missions to Honduras for regular monitoring of the program and overall economic and fiscal developments, and through a number of visits to Honduras by the Bank team. The ICR highlights that the team could have been more pro-active in the supervision of the fiscal situation that could have helped to better address the risks of deterioration.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

Although the Government played an important role in defining the reform agenda underpinning the program, Government’s overall performance deteriorated during program implementation. In addition to weak fiscal discipline that led to widening of Honduras domestic and external imbalances and cancelation of the second operation envisaged under the series, problems in tax administration had led to a decline in tax to GDP ratio.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:

The Ministry of Finance was the implementing agency coordinating the actions of the line ministries and other agencies involved in the operation. The ICR reports that the Ministry was pro-active in program implementation and provided leadership to address the bottlenecks.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The design of monitoring end evaluation framework suffered from weaknesses, especially in terms of choice and scope of indicators. The program had four outcome indicators that were not sufficient to fully capture the progress for a multi-sector operation. This was aggravated by weaknesses in some of the indicators. In particular, the indicator in tax administration (increase of the number of large taxpayers filing electronically) does not provide sufficient information about progress in tax collection. The ICR reports substantial increase in the number of large taxpayers filing electronically, while the overall revenue situation in tax administration deteriorated in 2012 leading to a decline in tax to GDP ratio. The Program Document, in the section on expected results, mentions that the program would lead to a 15.3 percent tax to GDP ratio. This is not reflected in program’s M&E framework, however.

    In the area of public wages, the target indicator of 0.7 percent decrease of public wage to GDP ratio as of end of 2012 against the baseline value of 11.0 percent in 2010 lacks coherence as the baseline value as of end of 2011 was already 9.8 percent (ICR of ERDPC, page iii). This is explained by the fact that public wage bill was brought down substantially in 2011 under the previous operation, and this was one of the main achievements of that operation. The ICR reports 9.7 percent wage bill in 2012, which suggests no significant decline in the wage bill during program implementation.
    The area of pension systems reforms lacks indicators that could allow assessing the impact of the reform program supported by the Bank. The Policy Matrix of the Program Document mentions a number of very useful indicators (such as net worth of pension institutions) as “indicative medium term indicators” that were not formally part of the Program's operational framework. These indicators could have served as outcome indicators because they are sensitive enough to major changes in pension plans in short-term. As for the outcome indicator in the area of citizen security, the ICR reports lack of data to assess program performance, raising questions about the relevance of this outcome indicator for monitoring of the program performance.

b. M&E Implementation:

The M&E arrangements placed responsibilities for collecting data on the Ministry of Finance.
M&E implementation was mostly adequate. The Ministry of Finance and the Bank had joint responsibilities in the preparation of periodic reports.

a. M&E Utilization:

The ICR does not report any utilization of the M&E system beyond monitoring program implementation

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
N/A

b. Fiduciary Compliance:
The ICR does not report fiduciary issues.

c. Unintended Impacts (positive or negative):

The ICR stresses that although there could have been some limited negative poverty impact because of strengthening of control over public wages, the reforms mostly concern households above the poverty line, and if successful will stabilize country's medium-term fiscal outlook, freeing resources for poverty reducing spending programs.

d. Other:
N/A



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Unsatisfactory
Despite reported achievement of three out of four outcome indicators, weaknesses in the design and M&E framework prevents full assessment of whether program objectives were met. Honduras experienced a decline in tax to GDP ratio, as well serious fiscal slippages during program implementation. This led to cancelation of the envisaged second operation of the series.  
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Unsatisfactory
Despite implementation of many important actions supported by the Bank, the Government failed to lay strong foundations for maintaining fiscal stability, including in sustaining tax flows and controlling fiscal expenditures.  
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

This Review supports the findings (called lessons) cited in the ICR, especially the need for addressing macro and fiscal stability issues in DPCs implemented in volatile environment through prior actions and outcome indicators. The review adds the following lessons:
    • It is important to address the unfinished agenda of previous policy lending operations that remain relevant for program objectives. This can contribute to a more successful operation.
    • If key conditions underpinning budget support operations, such as public expenditure management, are not in place or not being addressed by the program, the Bank would need to continue working on program preparation to ensure these conditions are met before approving the program.
    • Quality monitoring frameworks with adequate indicators are important ingredients for an operation's success not only because they enhance evaluability of the program, but they also help to guide the program toward success. Weak relevant indicators may undermine Government's focus on reforms.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR provides detailed accounts and analysis on most important aspects of the program. The ICR, however, does not report on the status of implementation of the triggers for the second operation which is important to understand the extent to which the Government was ready to move ahead with the second operation. The ICR does not reflect on the incoherence of some of the outcome indicators discussed in section 10. In addition, the ICR does not provide a rating for monitoring and evaluation.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6DPL-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions