|1. Project Data:
ICR Review Date Posted:
|Health System Modernization Project
Project Costs(US $M)
Loan/Credit (US $M)
|Health, Nutrition and Population
Cofinancing (US $M)
Board Approval Date
|Health (66%), Compulsory health finance (19%), Central government administration (14%), Sub-national government administration (1%)|
|Health system performance (67% - P)
financial management and procurement (33% - S)|
||ICR Review Coordinator:
|Hjalte S. A. Sederlof
||Robert Mark Lacey
||Christopher D. Gerrard
|2. Project Objectives and Components:|
According to the project Appraisal Document (PAD), the Project Development Objective was to:
According to the Development Credit Agreement (DCA), the objective was to:
- improve both physical and financial access to and the actual use of high quality primary health care services, with an emphasis on those in poor and under-serviced areas as well as to diminish the unnecessary use of secondary and tertiary care facilities,
- increase the effectiveness of the MOH [Ministry of Health] and HII [Health Insurance Institute] in formulating and implementing reforms in provider payments and health system performance, and
- improve governance and management in the hospital sector.
- improve the physical and financial access to, and use of, high quality primary health care services;
- improve the Recipient’s capacity to formulate and implement health policies and reforms in the health sector; and
- improve hospital governance and management.
The statement of PDOs in the two documents are consistent. The ICRR will assess the project on the basis of the PAD as being the more specific of the two PDO statements.
There were five key outcome targets:
At mid-term review, two key targets – numbers 4 and 5 – were dropped, and one new target was introduced, an increase in the number of primary care visits per capita per year.
- At least 70 percent of the population is enrolled with a primary health care provider and use him/her as their first source of health care;
- Reduced percentage of households who do not seek necessary health care because they cannot afford it and reduced share of household expenditure for primary care services;
- Increased patient satisfaction with Primary Health Care (PHC) treatment and improved outcomes for defined health conditions;
- HII is able to live within its overall budget, without additional allocations from the Ministry of Finance (MOF);
- Based on agreed key performance indicators, hospitals using new governance approaches perform better than those who do not.
b. Were the project objectives/key associated outcome targets revised during implementation?
Component A - Strengthening Sector Stewardship, Financing and Purchasing (estimated cost at appraisal - US$7.7 million; actual cost – US$4.7 million). This component would help the Health Insurance Institute (HII) to develop its capacity as the sole purchaser of health services, and would support capacity building to strengthen the stewardship roles of the Ministry of Health (MOH), the Institute of Public Health (IPH) and the Health Insurance Institute (HII). Activities would include (i) capacity building in the HII and its local branches; (ii) strengthening the policy formulation and performance monitoring functions within the MOH and the IPH; (iii) development of health information systems to support payment and management reforms; (iv) development and implementation of a system to monitor provider performance (both clinical and financial); (v) establishing a licensing/re-licensing scheme for physicians and health facilities, and an accreditation program for hospitals; (vi) development of health technology assessment capacity; and, (vii) building up MOH capacity in financial management, procurement and project coordination.
Component B - Improving PHC Service Delivery (cost at appraisal US$10.1 million; actual cost US$12.5 million). This component would support institutional reforms and limited investments aimed at improving quality of care among health care providers and in health facilities. The program would (i) facilitate registration of the population with the HII and enrollment with a primary care physician, together with related public information campaigns; (ii) build practice management capacity at the primary care level; (iii) develop and introduce clinical guidelines with an initial focus on primary care and the primary-secondary care interface and the most pressing health issues (e.g., child health, ante-natal care, respiratory infections); (iv) establish a continuing medical education system (CME) and link this to the re-licensing scheme; (v) retrain existing general practitioners and pediatricians in evidence based treatment of common conditions and rational drugs use, based on the clinical guidelines; (vi) provide basic equipment to physicians who complete the retraining program; and (vii) establish a grant facility to fund proposals from primary care providers in support of quality of care and continuum of care improvement initiatives.
Component C - Strengthening Hospital Governance and Management (cost at appraisal - US$1.3 million; actual cost 1.8 million). This component would provide initial steps to improve hospital operations and direction by focusing on (i) the development and introduction of accounting and internal control structures for hospital care providers and training in hospital management, (ii) developing the regulatory framework, including by-laws and regulations to support the move of MOH hospitals to the status of autonomous public entities; and (iii) piloting reforms of hospital management and governance structures in selected hospitals.
There were three level 1 restructurings during implementation:
The first restructuring in April 2010 (3 1/2 years after effectiveness) affected components B and C. In component B, a new national communications strategy and campaign was introduced; production of health identity cards for enrollment in primary care was financed; and the purchase of hospital equipment was included to strengthen the primary-secondary care interface. In the absence of strong Government commitment, proposed quality improvement grants were dropped from the component. In component C, privatization of all non-medical activities in regional hospitals was introduced.
The second restructuring in September 2011 included the dropping of software procurement and the registration of the population with the HII, as the design of the health insurance information system suffered from continuous delays (Components A and B, respectively). At the same time, pilot testing of the country’s hospital rationalization strategy was introduced as a new activity under Component C.
The third restructuring in February 2012 involved a second extension of the project’s Closing Date to June 30, 2012 to ensure successful completion of two big procurement packages on medical equipment.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project cost and financing: Project cost at appraisal was estimated at US$19.1 million. The actual project cost was US$19 million. Actual IDA financing was US$14.5 million; US$0.9 million was cancelled. The Borrower contributed US$2.89 million compared to an appraisal estimate of US$2.1 million. A Japanese PHRD Grant of US$1.61 million also contributed to the financing of the project.
Borrower contribution: The Borrower contributed US$2.89 million.
Dates: The project was approved on March 14, 2006 and became effective on September 7, 2006. The closing date was extended twice. The first extension of 18 months was granted as part of the first (May 2010) restructuring in order to complete: (i) the procurement of medical equipment for the regional hospitals; and (ii) the health insurance IT system, including health identification cards for ages 0-16 years old. The second extension of three months was granted in February 2012 to allow procurement of medical equipment to be completed. The project closed on June 30, 201 -- 21 months later than the original closing date of September 30, 2010.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:Relevance of objectives is rated high.
The project objectives are relevant to the country’s development priorities in health and to Bank country and sector assistance strategies. They directly support Albania’s long-term health sector reform strategy aimed at achieving equitable access to appropriate care in an environment of effective sector management. This is highlighted in Albania’s long-term National Strategy for Social and Economic Development. And the 2011-2014 CAS (Report No. 54188-AL) includes as one of its three strategic objectives to “[broaden and sustain] Albania’s social gains by improving access to better quality education and health services, and increasing the effectiveness of its social protection systems.” This is reflected in the objectives of the project under review as well as the programming of a Health Sector Project planned for FY2014, which will pick up on initiatives introduced under the project under review.
b. Relevance of Design:Relevance of design is rated substantial.
The design was relevant. It included a menu of activities that were consistent with project objectives. It drew on lessons learned from an earlier health project and from similar projects in neighboring countries. It included both primary care network expansion and rehabilitation, and challenging institutional reforms. Initially, institutional reforms that were being introduced did not appear to be well calibrated to absorptive capacity in the implementing agencies. This imbalance was to be addressed by external technical assistance that would gradually be replaced by local experts; and by local working groups to undertake coordination and technical management of the project. Overall project management was to be located in the MOH. While project design would not altogether withstand unanticipated shocks during implementation – repeated senior personnel changes in the MOH, including six Ministers, would lead to subsequent adjustments in project contents (the three project restructurings); and MOH would not turn out to have sufficient capacity to manage the project – key institutional reform activities were maintained. The results framework was inadequate: it included targets that were not achievable, and ones that were difficult to monitor correctly.
|4. Achievement of Objectives (Efficacy) :|
Objective 1: to improve both physical and financial access to and the actual use of high quality primary health care services, with an emphasis on those in poor and underserved areas as well as to diminish the unnecessary use of secondary and tertiary care facilities.
Efficacy is rated substantial. Physical and financial access to primary care showed measurable improvement, as did indicators of use and satisfaction, although concerns remained about the quality of care at health centers. The use of secondary and tertiary facilities declined (this may have been due to the introduction of performance-based payments and new clinical guidelines, as well as resulting from any outputs under this objective). There is no evidence on the effect on poor households.
- All regional hospitals were supplied with essential medical equipment;
- 2,200 primary health care facilities constructed, renovated and/or equipped;
- Training provided to 630 primary health care providers in family medicine, and to general practitioners in the implementation of new clinical guidelines;
- Quality training for primary care workers was provided in three regions, compared with a target of six regions
Objective 2: to increase the effectiveness of the MOH and the HII in formulating and implementing reforms in provider payments and health system performance
- The number of visits per capita per year to primary care centers increased from 1.6 visits to 2.0 visits, meeting the target.
- Based on the results of a household survey, patient satisfaction with treatment in primary care centers rose from 88.4 percent to 98 percent, However, there is no proper data to assess if outcomes for defined health conditions have improved as a result of project activities.
- The share of households not seeking necessary care due to financial factors fell by 47 percent to 19 percent compared to a target reduction of 20 percent.
- The share of the population enrolled with a primary care provider rose from zero to 70 percent over the project period, meeting the target.
- The number of pregnant women receiving antenatal care during a visit to a health provider increased from 95 percent to 97 percent, and the share of immunized children rose from 94 percent to 95 percent.
Efficacy is rated substantial.
In the absence of relevant indicators, there currently is no available evidence of increased effectiveness. However, the outputs (and intermediate outcomes) related to this objective can reasonably be expected to increase the effectiveness of the MOH and HII in formulating and implementing reforms in provider payments and health systems performance. The measures for institutional strengthening of the MOH are all aimed at improving health system performance. The strengthening of the HII as sole purchaser of health services establishes a provider payment system that, combined with a unified benefit package, provides a predictable and monitorable environment for the provision of primary care.
The Health Insurance Institute:
1. Management capacity was developed in HII to act as sole purchaser of health services;
2. A performance-based payment system and a unified benefit package were developed and applied to primary health care providers contracted by HII.
3. Hardware was procured for the development of an information system on participants in the HII scheme.
1. The establishment of HII as sole purchaser of health services has reduced fragmentation in financing and created the necessary environment for performance-based incentives for providers;
2. Performance contracts have been established with 420 health centers, 2,000 family practitioners, and 6,600 nurses;
3. The information system is being used for reporting/assessing primary care activities based on performance indicators.
The Ministry of Health:
1. A long-term health sector strategy was developed;
2. A monitoring and evaluation capacity was set up with responsibility for strategic planning, and monitoring and evaluation of the health system;
3. A licensing and accreditation system has been introduced for all providers;
4. Evidence-based clinical guidelines have been issued
1. MOH has improved capacity to monitor, implement and adjust the sector health strategy over time;
2. The introduction of clinical guidelines has resulted in the establishment of protocols in primary services;
3. All public health facilities have been accredited.
Objective 3: to improve governance and management in the hospital sector
Efficacy is rated modest.
A total of 230 hospital managers and other staff in tertiary and district hospitals were trained in general and financial management. A coherent national rationalization plan for hospitals was developed, including (i) strengthening diagnostic capacity in a limited number of regional hospitals and (ii) integrating others with primary care facilities; the plan, together with reforms in hospitals management, was piloted in two regions. A plan for outsourcing non-medical services in regional services was developed. And a framework was developed for hospital autonomy.
Again, there are no indicators of efficacy.
Efficiency is rated substantial.
The PAD (Annex 9) contained a cost-benefit analysis. The ICR also undertook a cost-benefit analysis. Due to difficulties in estimating Components A and C, both analyses focused on Component B – Improving PHC Service Delivery – covering about 50 percent of total project costs at the appraisal stage and 59 percent at project closing. Benefits were to be derived from better PHC services and a reduction in unnecessary hospital admissions and specialist services. The economic rate of return at appraisal was estimated at 12.9 percent and at 18 percent by the ICR. The difference is mainly generated by reductions in hospital admissions and referrals to specialists (minus 2.4 percent at appraisal vs. minus 5 percent at completion).
Project implementation suffered delays, requiring two restructurings including modification of some components and revisions to the Results Framework, and leading to an extension of the Closing Date by 21 months. All three components were modified, leading to a reduction in the base cost of the first component from US$5.02 million to US$4.41 million; an increase in the second component from US$9.07 million to US$10.06 million; and an increase in the third component from US$1.31 million to US$1.46 million. Total baseline costs fell from US$16.37 million to US$15.93 million.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated