Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Ve- Health Service Refor


  
1. Project Data:   
ES Date Posted:
06/30/2003   
PROJ ID:
P008215
Appraisal
Actual
Project Name:
Ve- Health Service Refor
Project Costs(US $M)
 108.0  87.2
Country:
Venezuela
Loan/Credit (US $M)
 54.0  50.5
Sector, Major Sect.:
Health,
Health and other social services
Cofinancing (US $M)
 0  0
L/C Number:
L3823      
   
Board Approval (FY)
  95
Partners involved
Prepared in coordination with IDB-financed project, which supports similar activities in different states, in support of Government's health program. 
Closing Date
12/31/2000 06/30/2002
         
Prepared by: Reviewed by: Group Manager: Group:  
Denise A. Vaillancourt
Ridley Nelson Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
The principal objective of the project was to improve health outcomes for the 3 million users of government health services in the states of Aragua, Falcon, Trujillo and Zulia. This was to be achieved by: (a) redefining key government health policies related to the organization and financing of government health services; (b) promoting decentralization by building the capacity of the four states to plan and manage health care in a decentralized framework; and (c) enhancing the quality and efficiency of government ambulatory clinics and hospitals by rehabilitating physical plant and replacing basic equipment. The project was also expected to contribute to poverty alleviation by improving the access of poor Venezuelans to good quality health care services.

b. Components
The project consisted of three Components. The Policy Analysis and Reform component was designed to support policy reform in the areas of: (a) hospital autonomy; (b) rationalization of the service delivery model; and (c) improved health financing. It was also meant to support other policy initiatives including development of a proposal for addressing health service delivery in Caracas and an assessment of health information needs and initiatives in the sector. The Institutional Development component was to build capacity at the state level to manage health care: by providing in-service management training to staff of state health directorates, district health offices, hospitals and ambulatory clinics; by developing and supporting systems for them to manage finances, human resources, strategic planning, materials, maintenance, waste and health information; and by supporting studies and specific institutional development projects identified by the states. The Strengthening of Health Service Delivery component was aimed at improving the quality and efficiency of health services in the four states by rehabilitating ambulatory clinics and hospitals and replacing and completing basic equipment.

c. Comments on Project Cost, Financing and Dates
Total final project cost was US$87.2 million or about 80% of the appraisal estimate of US$108. At US$50.5 million, total Bank financing fell somewhat short of the original appraisal estimate of US$54.0, but made up a greater share of the total final cost at 58% (vs. appraisal estimate of 50%). Despite two extensions of the original closing date, only 40% of planned clinic rehabilitations were implemented at a cost of double the original estimate for the entire component. Sharp cost increases were due to multiple factors, the most significant one being the excessive (and unchecked) expansion of the sizes of clinics (up to three times the size of original plans).


3. Achievement of Relevant Objectives:

Available data on health status in the four states supported under the project does not show trends remarkably different than those for Venezuela as a whole. Data for the states of Aragua and Trujillo reflect country-wide trends quite closely. Infant mortality rates declined in all states over the project period. Although declines were somewhat greater in Zulia and Falcon than in the country as a whole, it would be hard to attribute these differences to the project, as less than half of the planned infrastructure upgrading was implemented, and this with significant delays. Health data do register a notable increase in utilization in two of the four states (tripled in Aragua and doubled in Falcon). Although the reform policy studies were completed in four major reform areas, the overall quality of the studies was lacking, study recommendations were not adopted by the central and state governments and no pilot reform experiementation was launched. While the results of the studies conducted to develop alternative and more cost-effective models of health service organization were not implemented, the state of Aragua developed and piloted with loan financing a new model of care, which served as the basis for the design of the Integrated Care Model introduced by the Ministry in June 1999 for all states. The strengthening of capacity to plan and manage health care in a decentralized framework was only partially achieved through the staff training subcomponent, as no management system modernization took place. However, management capacity at the state level was strengthened thereby facilitating the process of decentralization. The improvement of health service delivery was achieved to some extent, but much less than the original project targets. In short, rather than achieving health sector reform, the project achieved health facility rehabilitation and some extension of coverage.

4. Significant Outcomes/Impacts:

Under the Institutional Development component, management training was successfully carried out to accompany the decentralization process. The number of staff trained exceeded appraisal estimates for the four states targeted under the project, providing health staff with basic management skills. The experience of designing and delivering the training programs, combined with cross-state support and exchange of experience, left states with enhanced capacity in the planning and organization of training programs. All of this contributed to improvement in staff morale. However, there are two important caveats to the the success of project training. First, enhanced capacity will be undermined unless measures are taken to reduce the high turnover of medical personnel, especially physicians. Second, the value and effectiveness of the training is not known because it was not evaluated. With project support, health infrastructure was rehabilitated and re-equipped thus fulfilling one important criterion for improved and expanded services in those four states.

5. Significant Shortcomings (include non-compliance with safeguard features):

Most of the studies carried out under the Policy Analysis and Reform component were of poor quality and their outputs were incomplete. Much of this is attributable to inadequate supervision of consultancies and inadequate follow-up, both by the Borrower and the Bank. Furthermore, delays in the availability of study results made them even less useful. None of the study recommendations was adopted by the central and the state governments, and no pilot reform experimentation was launched, thus denying the remaining project components of the necessary policy reform framework that was envisaged at the design stage. None of the management systems for the nine specialized management areas was developed with project financing. The project did, however, provide computer equipment in support of a management information system developed by one state with other financing. Sixty percent of rehabilitation and re-equipment of health infrastructure in the four states was not undertaken, due in part to significant cost overruns and to important delays in implementation.
6. Ratings:ICROED ReviewReason for Disagreement/Comments
Outcome:
UnsatisfactoryUnsatisfactory
Institutional Dev.:
ModestNegligibleWhile the training component provided decentralized staff with basic management skills, no reforms of the organization and financing of health services were developed or pilot tested. Such reforms were expected to provide the framework that would shape the institutional development and service delivery components. A rating of "negligible" is also consistent with Annex 5 of the ICR ("Ratings for Achievement of Objectives/Outputs of Components"), which rates ID negligible notwithstanding the modest rating given in the text.
Sustainability:
UnlikelyUnlikely
Bank Performance:
UnsatisfactoryHighly UnsatisfactoryOver and above the deficient design of the project, Bank performance during supervision of a non-performing project was seriously flawed, most notably: no formal MTR, no restructuring, infrequent supervisions, even less frequent visits to project states, unjustified PSR ratings, no updating/management of rapidly escalating infrastructure costs, no technical support/backstopping to policy reform component, and an unjustified QAG rating of supervision as "satisfactory" in 2000. The project outcome could have been significantly improved with an in-depth MTR and restructuring exercise and with more rigor, candor and proactivity during supervision.
Borrower Perf.:
UnsatisfactoryUnsatisfactory
Quality of ICR:
Satisfactory

7. Lessons of Broad Applicablity:

The following lessons are drawn largely from those presented in the ICR:
1. In countries with weak capacpity to design and impement complex health reform, it would be preferable to proceed with simplified and phased reforms coupled with management and infrastructure improvements, rather than preconditioning other components on the successful implementation of the policy analysis and pilot experimentation component.
2. Some key indicators to measure basic outcomes (such as increased coverage, quality and efficiency) should be agreed upon as part of appraisal and not left for development during project implementation. This might have reshaped the "Strengthenig of Health Service Delivery" component to include more than just hardware.
3. The preparation of policy studies should be carefully supervised. There should be active participation of beneficiaries and other stakeholders in the design of the studies and in the discussion of results. This is in stark contrast with the project design that envisaged activities to "strategically manage the image" of the Ministry in order to promote/defend policy reform after the fact.
4. Projects should be restructured when government policy and priorities change to ensure relevance and impact. The mid-term review exercise can be instrumental in assessing project design, implementation progress, and impact and in underpinning a restructuring exercise, if deemed necessary.
5. Bank supervision missions should visit project sites, assess progress on the ground, and be proactive and rigorous in reporting on and in responding to issues and constraints to good performance and impact. Supervision missions should also be seized as an opportunity to continue and deepen policy dialogue, especially in cases of projects with policy reform components.

8. Audit Recommended?  No

          Why?  

9. Comments on Quality of ICR:

The ICR is well organized, clearly written, analytic, well substantiated and candid. The only inconsistency found was the rating for Institutional Development, rated as "modest" in the main text, and as "negligible" in Annex 5.

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions