Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Health Services And Management Project


  
1. Project Data:   
ES Date Posted:
04/19/2001   
PROJ ID:
P008484
Appraisal
Actual
Project Name:
Health Services And Management Project
Project Costs(US $M)
 132.6  53.5
Country:
Hungary
Loan/Credit (US $M)
 91  35
Sector, Major Sect.:
Primary Health Including Reproductive Health Chi,
Health Nutrition & Population
Cofinancing (US $M)
   
L/C Number:
L3597      
   
Board Approval (FY)
  93
Partners involved
 
Closing Date
06/30/2000 06/30/2000
         
Prepared by: Reviewed by: Group Manager: Group:  
Roy Jacobstein
Timothy A. Johnston Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
This was the first Bank project in the health sector in Hungary. Project Objectives were to:

Contribute to improving the health status of the Hungarian population by supporting public health programs aimed at reducing risk factors contributing to the high prevalence of non-communicable diseases;
Support the Government's program of health sector restructuring with the aim of improving internal efficiency; and,
Provide more efficient and higher quality care by strengthening the institutional capacity of the health sector in policy-making, management and evaluation.

b. Components
There were two specified project components: Health Services Development and Policy-Making and Management. The first component, designed to "restructure the health sector to focus on more effective interventions," consisted of two subcomponents: Public Health and Disease Prevention, and Institutional Care. The second component, designed to strengthen the institutional capacity of the health sector, consisted of four subcomponents: Public Health and Management Training, Management Information Support Systems, Project Management (the PMU), and Preinvestment Studies.

c. Comments on Project Cost, Financing and Dates
Total project costs at appraisal were estimated at $132.6M, with a Bank loan of $91M, and GOH contribution of $41.6M. Latest estimates at closing were a total project cost of $53.51M, with a Bank loan of $41M, $6M of which was canceled at closure. The final estimate of GOH is $17.7M. HSMP was restructured twice. At MTR in 1996 a number of activities were cancelled due to unsatisfactory implementation progress, without loan cancellation or change in objectives. In 1998 upon the new Government’s review, the project was restructured and downsized, with $50M cancelled and benchmarks reduced but with overall objectives unchanged.


3. Achievement of Relevant Objectives:

Assessment of achievement of relevant objectives is difficult due to the near absence of quantifiable or operationally defined project goals, benchmark targets and performance indicators of achievement. As the ICR notes, "evaluation of the level of achievement is highly judgmental, since the original project design did not include monitoring indicators and mechanisms to measure performance...in addition, the three development objectives are too general...mere rhetoric." Within various components and subcomponents the project did have some successful achievements, as noted in Section 4. But on the whole few project activities are likely to be sustainable and even fewer outcomes can be documented to have led to achievement of the stated objectives, given their generality.

4. Significant Outcomes/Impacts:

The project's main contribution was in the areas of institutional development and program management. HSMP led to the establishment of a School of Public Health, which has seen two cohorts of classes graduate. The Project also developed a Health Services Management Training Center and supported overseas training in PH and/or management. According to the MOH, 10 students had submitted a thesis as of 6/00 and 68 more were studying in the SPH. Similarly, 26 students have received an M.Sc. diploma in health services management and 26 more are currently studying for one. An additional 44 students have completed Master’s Degrees abroad in HSM. Finally, 21 hospitals (13%) were equipped with a basic MIS. This activity area was singled out in the ICR as one that survived despite many obstacles (turnover of staff and governments), and "could serve as a best practice." However the specific accomplishments in measurable terms of efficiency and impact (outcomes, not process) are not given. The project helped establish a school health program, and also strengthen the national cancer register and screening programs for breast, cervical, and colon cancer.

5. Significant Shortcomings (include non-compliance with safeguard features):

HSMP suffered from a number of major shortcomings. Project design lacked clearly defined objectives, measurable performance indicators and benchmarks for achievement, and a monitoring and evaluation plan. It was too ambitious, expensive and complex (29 separate activity clusters in the original design), particularly for a first project. The health sector was not a GOH priority, there was no GOH strategy to which subcomponents could be linked. There were too many consultants on the project, and the PCU was not closely enough linked to policymakers. GOH "ownership" was weak throughout much of the project. Although project design placed a strong emphasis on prevention of noncommunicable disease, efforts to reorient the National Public Health and Medical Officer's Service were largely unsuccessful, and the ministry of health showed limited commitment to health promotion and preventive services.

6. Ratings:ICROED ReviewReason for Disagreement/Comments
Outcome:
UnsatisfactoryModerately UnsatisfactoryDespite an overly complex design and low borrower commitment, the project made some specific contributions, particularly in establishing new training programs, that may yield future benefits.
Institutional Dev.:
ModestModest
Sustainability:
UnlikelyUnlikely
Bank Performance:
SatisfactorySatisfactoryAn unsatisfactory rating could be justified based on shortcomings in project design and unsatisfactory outcome. The Bank was proactive in supervision, however, including two restructurings of the project, and provided active support from the resident mission.
Borrower Perf.:
UnsatisfactoryUnsatisfactoryThe borrower's ICR concurs with a ratings of "unsatisfactory" for borrower performance, but "satisfactory" Bank performance.
Quality of ICR:
Satisfactory

7. Lessons of Broad Applicablity:

Borrower commitment to a project’s strategy, goals, and implementation, i.e., "ownership," as manifested in senior-level involvement in design, and in sector strategies, policies, and programs to which the project links, is critical to successful project performance; in its absence, a project should not proceed.
Health sector projects should avoid overly complex design, with their size commensurate with counterpart capacity.
Project components and subcomponents should devote a greater share of funds and effort to human resource development ("the software of development"), with less emphasis on purchase of equipment ("the hardware of development").
Projects attempting to address the problem of skewing of resources toward curative care (away from preventive care) should not then devote a substantial percentage of project resources to curative care, whatever the pressures to do so.
Baseline indicators and interim benchmarks for project performance, in terms of both outputs and health outcomes, need to be identified at appraisal, and regularly tracked and commented upon at supervision, as well as used in final evaluation.

8. Audit Recommended?  Yes

          Why?  Medium priority, as part of a possible sector study. This could also allow an assessment of the sustainability of various project activities.

9. Comments on Quality of ICR:

The ICR was well-written and it adequately addressed the problem of describing, analyzing and assessing a complex and not particularly well-articulated or well-designed project. The ICR was frank in its assessments--many of the components and activities were judged to be unsuccessful, and with several exceptions the borrower essentially agreed with the ICR assessments. The lessons learned section is particularly robust. The borrower's ICR argued that a more positive overall assessment of project outcomes -- e.g., while project accomplishments fell short of initial objectives, it still made useful contributions. As part of the ICR process, the Bank financed a visit by several Hungarian officials to Estonia -- which was also completing an ICR for a project that was highly successful. This represents a "best practice" example of leveraging cross-country learning and policy dialogue as part of the ICR process.

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions