Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Kh-health Sector Support Project


  
1. Project Data:   
ICR Review Date Posted:
05/09/2013   
Country:
Cambodia
PROJ ID:
P070542
Appraisal
Actual
Project Name:
Kh-health Sector Support Project
Project Costs(US $M)
 31.84  34.29
L/C Number:
C3728, CH015, CH016
Loan/Credit (US $M)
 28.84  32.06
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 3.00  2.23
Cofinanciers:
UK: Department for International Development (DFID)
Board Approval Date
  12/19/2002
 
 
Closing Date
12/31/2007 12/31/2011
Sector(s):
Health (80%), Non-compulsory health finance (10%), Sub-national government administration (5%), Central government administration (5%)
Theme(s):
Tuberculosis (23% - P) Nutrition and food security (22% - P) Health system performance (22% - P) Child health (22% - P) Rural services and infrastructure (11% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Moritz Piatti
Judyth L. Twigg Ismail Arslan IEGPS2

2. Project Objectives and Components:

a. Objectives:


    According to the Project Appraisal Document (PAD, p. 3) the development objectives were:
        "to contribute to the improvement of the health status of the population by: (a) increasing the accessibility and the quality of health services; and (b) assisting the Kingdom of Cambodia to implement its Health Sector Strategic Plan and strengthen the sector's capacity to manage resources efficiently."

    According to the Development Credit Agreement (schedule 2, p. 16) the objectives of the project were:
        "to assist the Kingdom of Cambodia to improve the health status of its population, particularly the poor and rural population, through: (i) improvement in the accessibility, quality and affordability of health services in selected Provinces, and (ii) support for the implementation of its Health Sector Strategic Plan and strengthening of its capacity to carry out the health sector reform and to manage the health sector resources efficiently."

    The objective of the DCA is used for this review as it points to the health outcomes of the poor and rural population.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

Component 1: Improved Delivery of Health Services (appraisal, US$ 14.41 million; actual, US$ 15.70 million)

Component 1 was to focus on access, quality and affordability by financing the development of primary health care facilities, referral hospitals (at district and provincial levels), equipment, and maintenance. It was also to support training to improve service quality, the establishment of a Quality Improvement/Standards Unit in the Ministry of Health (MOH), and quality assurance activities in three districts in the Kampong Thom province. Other planned activities were performance-based contracting for district health services, activities to increase user participation in decision-making, health equity funds to pay for services provided to the poor, and improvement of drug quality and availability.

Component 2: Improved Programs Addressing Public Health Priorities (appraisal, US$ 10.21 million; actual, US$ 12.05 million)

    Component 2 was to focus on infectious disease control (i.e. malaria, tuberculosis, dengue and sexually transmitted infections/HIV/AIDS, and nutrition). For malaria, support was to include the provision of insecticide treated bed-nets, training for better case management, improved surveillance, and information, education and communication activities. TB control activities were to include integration of activities at the Health Center (HC) and district hospital levels, increasing case detection and reducing default rates, improving laboratory capacity, and strengthening information, education and communication activities. For dengue control, the activities were to include strengthening early diagnosis, appropriate treatment, and vector control. For STI/HIV/AIDS, the project was to support procurement of drugs and 100% condom use among high risk groups. For nutrition, financing was to include training, information campaigns, and community outreach to support an essential package of preventive and curative services, such as exclusive breast-feeding up to 6 months, timely and adequate complementary feeding from 6 to 24 months, appropriate care of sick and malnourished children, micro-nutrient supplementation for women and children, and increased availability of iodized salt.
    Component 3: Strengthened Institutional Capacity (appraisal, US$ 4.05 million; actual, US$ 6.54 million)
    Component 3 was to provide support to key functions at central, provincial and district levels. This was to include oversight of policy, improving the legislative and regulatory framework, better sector planning and program coordination (specifically improving Ministry of Health analytical capacity for health financing, planning capacity at provincial and district levels, and coordination and monitoring of plan implementation). The component planned to improve sectoral management, including human resource planning and management, improvement of skills and performance, strengthening financial management systems and capacity, and recruitment of procurement consultants. The component sought to improve monitoring and evaluation (M&E) capacity, including surveillance of health service delivery outside the public sector, and the use of data to inform health sector governance and planning.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost

Actual total project costs were 7.7% higher than anticipated. The ICR does not provide information on reasons for the increased project cost.

Financing

The Cambodia Health Sector Support Program (HSSP) was designed and financed by the International Development Association (IDA), the Royal Government of Cambodia (RGC), the Asian Development Bank (ADB), and the UK Department for International Development (DfID). After inception they were joined by the United Nations Population Fund (UNFPA). Whereas DfID channelled funds through the World Bank Trust Fund, UNFPA and ADB funding was managed in parallel. Though UNFPA and the ADB had their own reporting systems, they joined common management arrangements under HSSP, which was seen as the first step towards a sector-wide approach (SWAp) (ICR, p.1).

The project was financed through an IDA credit of US$ 18.99 million, an IDA Grant for Poorest Country of US$ 8.96 million, an IDA Grant for HIV/AIDS of US$ 2.24 million, and a DFID Trust Fund grant of US$ 1.32 million (ICR, p. 25).

Borrower Contribution

Against a planned US$ 3.00 million, the borrower contributed US$ 2.23 million. The Borrower's contribution was lower than anticipated because the World Bank agreed to increase the percentage of Bank financing for (i) works under categories A of the project from 90% to 100%; and (ii) consulting services under parts A and C of the project from 95% to 100%. In addition, a Health Equity Fund (HEF) was added with 50% counterpart financing, but the reduction in counterpart financing for other components offset this increase.

Dates

The project closed 4 years and 6 months later than initially planned. The original closing date of 12/31/2007 was extended to 12/31/2008 due to start-up delays and low disbursements. Two further extensions to 12/31/2010 and 21/31/2011 were granted to enabled the project to complete construction activities that had been subject to delays (ICR, p. 3).


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

High. The development objectives were highly relevant to country conditions, Government policy, and the Bank's strategy at the time of appraisal, and they remained relevant at closure.

At the time of appraisal service utilization was inequitable, and access and affordability of services for the poor were seen as critical. Many health indicators were below the average of Southeast Asian Countries, with particular concern for communicable and infectious disease, malnutrition, and maternal and child mortality (PAD, p.9).
The Bank's most recent Country Partnership Strategy (2005 - 2008) contained a focus on equity in access to services and improving quality and efficiency of services to the poor (CPS, p. 35). The current Government Health Sector Strategic Plan (2008 -2015) renders the overarching objective of improving the health status of the population as very relevant, as addressing population health problems is termed a strategic priority. Similarly, improvements in accessibility, quality and affordability are framed as priorities (HSSP, p. 2). Capacity strengthening and managing resources efficiently are also mentioned.

b. Relevance of Design:

Substantial. The Project Appraisal Document's results chain is plausible. The development of health care facilities and support to infectious disease control programs credibly feed into the objective of improved health status of the population. Similarly, a focus on primary health care facilities feeds into the objective of improved access to health services (especially for the poor), and the support of health equity funds is likely to contribute to the affordability of health services to the poor. The establishment of a Quality Improvement/Standards Unit within the Ministry of Health (MoH) and quality assurance activities in selected districts are likely to have quality improvement implications. Finally, improved capacity to carry out health sector reform and manage the health sector efficiently are conceivably achieved through a component that focuses on improved monitoring and evaluation (M&E) capacity and better sector planning and program coordination.

The results framework captures well how inputs are intended to translate into outputs and outcomes, though it would have benefitted from more detail in measuring accessibility, quality, and improved capacity to carry out health sector reform and manage health sector resources efficiently.


4. Achievement of Objectives (Efficacy) :

The Project Development Objective (PDO) is broken down here into 5 parts: (1) to improve the health status of the population, particularly the poor and rural population; (2) to improve the accessibility of health services in selected provinces; (3) to improve the quality of health services in selected provinces; (4) to improve affordability of health services in selected provinces; and (5) to strengthen the capacity to carry out health sector reform and manage the health sector resources efficiently.

Some data in the ICR's cover datasheet and the discussion of key/core performance indicators in the main ICR text are conflicting, in which case the datasheet was given precedence.


    Objective (1): To improve the health status of the population, particularly the poor and rural population:

    Additional evidence presented by the region renders this objective Substantial.
    Outputs:

    The pulmonary TB smear (+) case detection rate increased from 57% in 2002 to 66% in 2011, missing the target of 70%.

    The pulmonary TB smear (+) cure rate decreased from 93% in 2003 to 91% in 2011, meeting the target of >85%.

    The percentage of children under 1 year of age fully immunized was not measured. Their immunization for measles increased from 39% in 2000 to 73.6% in 2010, surpassing the target of 70.0%.

    The percentage of pregnant women receiving at least two antenatal care consultations increased from 29% in 2002 to 72% in 2010, nearly achieving the target of 75%.

    The share of rural women receiving antenatal care increased from 25% in 2000 to 79% in 2010. No target was set.

    The percentage of deliveries attended by trained staff increased from 20% in 2002 to 69.6% in 2010, nearly meeting the target of 70%.

    The share of rural women delivering in health facilities increased from 11% in 2000 to 60% in 2010. No target was set.

    The percentage of married women aged 15-49 years using modern contraceptives in public health services increased from 17% in 2002 to 35% in 2010, falling short of the target of 44%.


    Outcomes:

    The HIV sero-prevalence rate among women attending antenatal care dropped from 1.9% in 2002 to 0.4% in 2010. No target was set.

    The malaria case fatality rate of severe cases treated in public facilities per 100 patients decreased from 10.85% in 2002 to 0.22% in 2011. A target of 0.25% was set for 2010.

    The incidence of malaria per 1000 inhabitants in high risk areas was reduced from 8.6 in 2002 to 3.34 in 2010, meeting the target of 3.58 in 2010.

    The national average maternal mortality ratio measured in terms of live births increased from 437/100,000 in 2000 to 472/100,000 in 2005 and subsequently decreased substantially to 206/100,000 in 2010, exceeding the target of 243/100,000 live births.

    The national average fertility rate decreased from 4.0 in 2000 to 3.4 in 2005 to 3.0 in 2010, exceeding the target of 3.5 in 2010. No disaggregated information is provided for the rate of the rural poor targeted under this project.

    The national share of stunting among children under 5 years of age decreased from 45% in 2000 to 39.9% in 2010, missing the target of 22%. Rural stunting decreased by 8 percentage points from 51% in 2000 to 43% in 2010. No target was set.

    The national share of wasting among children under 5 years of age decreased from 15% in 2000 to 10.9% in 2010, missing the target of 9%. Rural wasting among children under 5 years of age decreased by 8 percentage points from 16% in 2000 to 8% in 2010. No target was set.

    The national percentage of children under 5 years of age who were underweight decreased from 45% in 2000 to 28.3% in 2010, meeting the target of 29%. The share of rural children under the age of 5 who were underweight was reduced by 9 percentage points from 41% in 2000 to 32% in 2010. No target was set.

    The national mortality rate of children under 5 years of age decreased from 124/1,000 in 2000 to 54/1,000 in 2010, exceeding the target of 85/1,000. Child mortality in rural areas decreased from 18.1/1000 live births in 2000 to 4.9 live births in 2010. No target was set.

    The national infant mortality rate per 1000 live births was reduced from 95 in 2000 to 45 in 2010, exceeding the target of 60. The infant mortality rate per 1000 live births in rural areas was reduced by 44% from 96 in 2000 to 54 in 2010. No target was set.

    Child mortality in the poorest quintile declined from 127 per 1000 in 2005 to 90 per 1000 births in 2010, compared to a decline from 43 to 30 per 1000 among the richest quintile. This is an approximate 30% reduction in both groups.


    Objective (2): To improve the accessibility of health services in selected provinces: Substantial

    Outputs:

    The number of health centers offering Minimum Package of Activities (MPA) services was 294 in 2003, and a target of 470 was set for 2006. In 2010, 469 out of 1010 offered the MPA.

    In 2011, 56 referral hospitals offered the Complementary Package of Activities (CPA), against a target of 46 in 2006.

    The per capita consultation rate in public facilities of new cases increased from 0.38 in 2002 to 0.63 in 2010. The target set for 2006 was 0.5.

    Outcomes:

    Average distance for the poorest quintile to travel to a health center is reported to have fallen by 36% between 2004 and 2007 (Cambodia Social and Economic Survey 2004, 2007). The ICR does not provide further detail on this trend.
    Objective (3): To improve the quality of health services in selected provinces: Modest

    Outputs:

    A Quality Assurance Office was established and is still functioning in the Ministry of Health (MOH), and various quality assessment tools were developed and implemented.

    Outcomes:

    No consistent methodology was adopted to measure client satisfaction. In 2005, the percentage of patients satisfied with services received in public health facilities was 95% for the poorest and 89% for the better off. Satisfaction at public facilities was rated 8.4/10 in 2010 (ICR p. 41).

    A study on the lessons of Health Equity Funds (HEFs) in Cambodia (WHO bulletin 2009) finds that HEF patients did not face stigma, were rarely charged unofficial fees, and helped improve overall quality of care. It is argued that these Funds address quality of care issues by providing additional facility revenues and establishing formal contractual arrangements. HSSP in total supported 13 HEFs.

    The ICR provides no data comparable across time on patient satisfaction, or other evidence on quality of health services.


    Objective (4): To improve affordability of health services in selected provinces: Substantial

    Outputs:

    The number of fully and partially exempted households for which costs have been paid by equity funds increased from 18,591 in 2003 to 227,457 in 2008. No target was set.

    Outcome:

    The percentage of people in the poorest quintile paying catastrophic out-of-pocket expenditures for health care was reduced from 3.6% in 2004 to 2.3% in 2009. No target was set.

    An independent evaluation of the Health Sector Support Project (HSSP) commissioned by the MOH undertook a household survey and beneficiary assessment on Health Equity Funds (HEF), which concluded that HEFs reduce out-of-pocket and catastrophic spending by the poor on health (ICR, p.12). The ICR provides no information regarding the size of the impact or performance in comparison to control groups.

    Objective (5): To strengthen the capacity to carry out health sector reform and manage health sector resources efficiently: Modest

    Outputs:

    The percent of Provincial Health Departments (PHD) producing annual health plans remained at 100% from 2004 to 2008, reaching its target of 100%.

    The percent of Operational Districts (OD) producing annual health plans remained at 100% from 2004 to 2008, reaching its target of 100%.

    The MoH recurrent budget (salary excluded) as a proportion of the total government recurrent budget increased from 9.5% in 2003 to 11.2% in 2008, missing the target of 11.9%.

    The project team later provided further information: with project funding, the MOH introduced a rolling mid-term planning process and redefined the planning roles of the central, provincial, and district levels. The project supported the progressive decentralization of the planning process. Further, the project contributed to the organization of annual sector reviews, which brought together stakeholders to review sectoral achievements and discuss the comprehensive rolling plan.
    Outcome:

    Although there were activities implemented in this area, little information is provided in the ICR or in the additional information provided by the project team on improvements in the actual capacity to carry out health sector reform and manage health sector resources efficiently.

5. Efficiency:

The PAD (p. 67) provided an economic analysis of the project including an estimated internal rate of return of 17% and a net present value of US$ 45 million. This return was seen as not robust in the face of moderate project risks without the added benefits from the priority programs.

No follow-up analysis was provided by the ICR.

Public sector reform and project investments were largely delayed, including the rebidding of civil works contracts that necessitated a further two-year extension. The rebidding process, however, resulted in 30% lower contract prices.

The ICR provides no further detail on implementation at least cost or other value-for-money analysis.

Overall efficiency is rated as modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
17%
100%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


    The importance of the project given the country circumstances and sector priorities, as well as a plausible results chain, renders relevance of objectives and design as high and substantial respectively. Achievement of the overarching objective of improving health status, particularly among poor and rural populations, is rated substantial. Improvements in affordability were recorded, though not matching the years of the project. There were only modest improvements in the quality of health services and the capacity to carry out health sector reform and manage sector resources efficiently. Efficiency is rated modest due to delays in implementing project investments.

    a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The government has shown commitment to health by increasing the MOH share of overall recurrent budget (excluding salaries) from 9.5% to 11.2%. Furthermore, the government continues to demonstrate political commitment to reducing the MDGs as reducing HIV prevalence, maternal and child mortality, and poverty have been given strong emphasis in government policies.

The government has also shown commitment to reform in public sector financial management and decentralization. However, it remains unclear that the government's capacity to carry out health sector reform and manage health sector resources efficiently was strengthened significantly by the project, which may prove a liability in the long run.

The HSSP was designed to use MOH systems and was fully endorsed by the MOH during the design stage. However, the Bank subsequently agreed to a MOH request to establish a Project Management Unit (PMU) to facilitate implementation. Though this may have reduced fiduciary risks, it also meant that the procurement function was not transferred to the Government, and the project's contributions in terms of capacity building, ownership, and sustainability were mitigated.

A follow up project (KH - Second Health Sector Support Program, approved in June 2008, US$ 30 million) was built on lessons from HSSP1. The transition commenced smoothly in early 2009, when most of the operations of HSSP1 other than construction had already closed.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

A June 2003 quality of entry assessment by the World Bank Quality Assurance Group rated overall quality of entry Marginally Satisfactory. The preparation of the HSSP took into account prior experience of the World Bank (most notably the Cambodia Disease Control and Health Development Project (Cr. N005-KH)) and other donors in the health sector. This prior experience indicated that the capacity of the MOH was weak, especially in the areas of procurement, financial management, and planning, and the project was therefore intended to strengthen these areas.

Analytic work during project preparation included: (1) an analysis of poverty and health; (2) a review of mechanisms to improve equity of access to health services; and (3) beneficiary and social assessments. The design correctly identified the major disease burdens in the country, weaknesses in health system management and monitoring, issues of accessibility and coverage, and the problem of access for the poor following the introduction of user fees in 1996.

The overall risk rating of the project was Substantial (PAD, p. 44). The major risks identified were financial management capacity, political will for reform, planning capacity, commitment to human resource development, low government salaries, slow release of government funds, and weak lateral cooperation within the MOH. The respective mitigation strategies included the opening of Advance Accounts at provincial levels, annual reviews focusing on progress in particularly difficult areas, institutional strengthening in the areas of planning, procurement, and disbursement, substantial additional support for the implementation of the Health Workforce Development Plan (1996 - 2005), salary supplements, training in financial management and accounting, and strengthening coordination capacity through the implementation of the Project Secretariat. Risks were considered as correctly identified and the mitigation strategies as appropriate (ICR, p. 4).

In order to harmonize the HSSP with the Health Sector Strategic Plan 2003-2007 (HSP1), common management, monitoring, and reporting mechanisms were agreed upon. However, HSP1 was not finalized when the HSSP was negotiated, and the adoption and reliance on the HSP1 M&E framework for HSSP meant that there were deficiencies, as some indicators remained immeasurable. These included the percentage of facilities meeting essential obstetric care standards, the population share with access to facilities providing a Minimum Package of Activities (MPA) services, the percentage of hospitals meeting minimum staffing levels, and the percentage of facilities without drug stock outs. The baseline and target values were not always set to overlap with project implementation, precipitating ambiguity of attribution. The M&E framework allowed for only limited tracking of progress on the objectives of accessibility, quality, and strengthening capacity to carry out health sector reform.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:

    The Joint Review Mission (JRM) produced detailed aides memoire with clear recommendations and action plans for all partners and the MOH. Participation in annual planning workshops helped to maintain a focus in the MOH on using evidence to identify and address priorities.

    There were some delays in release and acquittal of funds, procurement, and resolving issues in the PMU. The placement of a senior health specialist and local staff in the country helped to reduce delays and resolve problems more quickly. Frequent meetings with MOH and the PMU meant issues were identified earlier and mitigation measures agreed in a more timely way.

    However, the M&E framework was set up such that it remained difficult to track progress on the objectives of accessibility, quality, and strengthening capacity to carry out health sector reform. Supervision did not update the results framework appropriately to better track progress, or to revise the project development objectives to better align the project's ambitions.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The government showed commitment to reforms in public sector financial management, decentralization, and localized planning in the lead-up to HSSP. The Ministry of Economy and Finance (MEF) was supportive of the internal and external audit process for HSSP.

The government remains strongly committed to the MDGs. Reducing HIV prevalence, maternal and child mortality, and poverty have been given strong emphasis in government policies, and clear interim and final targets were set in these areas, which were often used as targets for project performance indicators.

The government has shown interest in SWAps in order to improve donor harmonization and alignment with government systems. However, more needed to be done in terms of consultations with local government and civil society in the context of decentralization and linkage to other social protection measures.

The MEF assigned staff to liaise with and monitor the project. It also oversaw use of guidelines on financial management and procurement, took part in joint portfolio reviews, and trained MOH and PMU staff when there were changes in government policies.

The detection of irregularities in procurement in ministries other than the MOH triggered the government to take the decision to mandate all World Bank-financed projects to use services of the International Procurement Agency (IPA) for procurement. Though this may have helped reduce fiduciary risks, it contributed to delays for procurement of civil works of HSSP. This also meant that it was not possible to transfer the procurement function from the PMU to the government system, which was initially planned. In addition, the government showed little commitment to strengthening the national procurement system, and overall public financial management reform implementation was slow.

There were delays in release of the national budget, especially in the first quarter of each fiscal year, which was due to the government budget process.

MEF monitored financial management and procurement issues, took part in reviews of audits of the project, and was active in trying to resolve procurement problems.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:


    The Cambodia Ministry of Health was the implementing agency, and implementation took place under considerable capacity constraints. The health sector was given a high profile with regards to the MDGs, and the MOH was strongly committed to the project in part because it offered a way to make progress on these issues. The MOH showed particular interest in increasing access and improving quality and affordibility of service delivery while sustaining user fees for the non-poor. HSSP support for HEFs was considered a systematic and sustainable way to confront the problem of user fees for the poor.

    The MOH set up a steering committee and financial management group, with a project director at a sufficiently high level for decision making and political advocacy. Meeting effectiveness conditions took longer than expected, but once HSSP became effective, activities began quickly across all components.

    The MOH interacted regularly with donors and NGOs through the Technical Working Group for Health, which convened monthly. Consultation and discussion with provincial and operational districts was most active during the annual planning process and mid-term reviews. With regard to decentralization, local governments could have been consulted more.

    While no serious issues were identified during regular audits, MOH and PMU personnel were often slow to respond to findings, with problems often carrying over from one audit to another. Procurement, in particular of goods and civil works, experienced delays, which necessitated the extension of the project's closing date and the transfer of unfinished works to the follow-on project. The heavy workload in procurement was complicated by changes in government policy that also contributed to the delays.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:


    The project was prepared in parallel with the preparation of the Health Sector Strategic Plan 2003-2007 (HSP1), which facilitated alignment of project design with MOH strategic priorities. In the spirit of harmonization, it was agreed to join common management arrangements and rely on MOH systems for planning, program implementation, monitoring, and reporting.

    However, the HSP1 monitoring and evaluation (M&E) framework was not finalized when the project was negotiated, and some indicators included in the PAD did not have clearly defined baselines, targets, or means to collect data. In particular this left the following indicators unmeasured: (1) percentage of facilities meeting essential obstetric care standards; (2) the population share with access to facilities providing a Minimum Package of Activities (MPA) services; (3) the percentage of hospitals meeting minimum staffing levels; and (4) the percentage of facilities without drug stock outs. The baseline and target values were not always set to overlap with project implementation, precipitating ambiguity of attribution.

    In addition, the M&E framework allows for only limited tracking of progress on the objectives of accessibility, quality, and strengthening capacity to carry out health sector reform.

b. M&E Implementation:

    M&E indicators as set out in the HSP1 were monitored regularly by the MOH. Health Management Information System (HMIS) data were used in monitoring sector annual progress, and data timeliness improved over time. These data were supplemented by the Cambodia District Household Survey (CDHS) for some key indicators. HSSP indicators using HSP1 were thus updated annually using the HMIS, and those relying on the CDHS used results from 2005 and 2010 as available. However, reliance on these sources also meant that some indicators as outlined in the PAD were not measured.

    Furthermore, the M&E framework was not updated appropriately to reflect progress on some of the objectives. There is particularly little evidence on improvements in accessibility of health services, quality of health services, and strengthened capacity to carry out health sector reform.

a. M&E Utilization:

Project support to strengthen the MOH's health management information system (HMIS) improved data quality and timeliness, which continued under the follow-on project. HMIS data were regularly used in annual planning workshops and annual joint reviews of progress by the MOH, donors, and NGOs. The use of data by provinces, operational districts, and health facilities improved during implementation; for example, health facilities now post monthly progress on key indicators near their entrances.

Data from the CDHS were used to assess progress towards the MDGs, analyze health spending and service utilization by the poor, and facilitate broader policy development and decision making. Furthermore, lessons learned during HSSP1 fed into the design of HSSP2.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
Project design triggered safeguards related to the environment (OP 4.01), pest management (OP 4.09), indigenous peoples (OD 4.20), and involuntary resettlement (OP/BP 4.12). The Environmental Review classified the project as a Category B activity, where limited environmental analysis is considered appropriate to address specific environmental issues.

With respect to the environment, the MOH adopted a medical waste management plan, which was monitored by a Bank environmental specialist at least annually. National guidelines on infection control were developed, training was provided for health staff at all levels, medical waste was properly segregated before disposal, safety boxes were used for collecting sharp waste, and incinerators were constructed as part of facility construction or upgrading.

All insecticides financed by the project were pre-qualified by the WHO.

An assessment of environmental and health impact was undertaken prior to the construction of the National Laboratory Quality Control Laboratory.

With respect to social safeguards, an indigenous peoples consultation was done as required by the project. The Bank and the MOH collaborated on a sample verification of compliance with resettlement safeguards, which were found to be satisfactory. The overall rating was Moderately Satisfactory.

b. Fiduciary Compliance:

    Financial management monitoring was discussed at every Project Joint Review Mission and recorded in aides memoire and ISRs. The financial management rating up to 2004 was Satisfactory but downgraded in June 2005 to Moderately Satisfactory upon country arrival of the TTL, where it remained until project closure.

    The procurement rating was downgraded from Satisfactory to Moderately Satisfactory in June 2005 and further to Moderately Unsatisfactory in 2009 and then Unsatisfactory in March 2011. Upon successful rebidding of civil works contracts and improved procurement implementation, the final ISR restored the rating to Moderately Satisfactory.
    The ICR does not discuss why procurement ratings were continuously downgraded. The heavy workload in procurement was complicated by changes in government policy that contributed to delays. MEF monitored financial management and procurement issues, took part in reviews of audits of the project, and was active in trying to resolve procurement problems.

    The ICR does not state whether audits were on time and unqualified. While no serious issues were identified during regular audits, MOH and PMU personnel were often slow to respond to findings with problems often carrying over from one audit to another.

c. Unintended Impacts (positive or negative):

None noted in the ICR.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory

 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
    The ICR (p. 21) provides several lessons, adapted her by IEG:

    It is important that a technical sectoral analysis is matched with a political economy analysis of government systems and structures in order to help clarify risks and select appropriate mitigation strategies. The project suffered from delays in implementation, which often arose because of systemic issues in the partner government that were not fully appreciated or taken into account during project design and could have been anticipated via a thorough sectoral analysis.

    Innovative management solutions can considerably improve project efficiency, especially in SWAps. In this case, direct disbursements of funds from the PMU to provincial accounts sped up the financial management process , bypassing bureaucratic bottlenecks. The challenge is convincing governments to adopt more efficient methods into existing systems.

    Implementation can improve when World Bank supervision switches to in-country management. In this case, in-country presence meant that regular meetings to discuss strategy, resolve problems, and build relationships with government and other stakeholders became feasible and convenient. It also meant that assessments of progress were more realistic because of the day-to-day involvement.

    In addition to the above lessons from the ICR, IEG draws an additional lesson from M&E performance. A solid monitoring framework, which tracks progress on the objectives and alerts the implementing team if there is a need to revamp the results framework, will have an critical impact on performance.

14. Assessment Recommended?

Yes
Why?
To elaborate on the contribution of the HSSP to improved health outcomes in Cambodia in an environment where factors other than public health expenditure could have played a major role.

15. Comments on Quality of ICR:


    The ICR provides a clear assessment of the project's achievements, although it would have benefited from a more detailed analysis and analytical discussion of the contribution of the outputs to the expected outcomes. Additional information on efficiency (especially revisiting the IRR with actual data), detailed reasons for extensions of closing dates, information on reduced borrower contributions, and information on audits are lacking. The ICR does not clearly present baseline, target, and actual indicator data, and on occasion different figures for the same indicator are provided at different points in the document. Furthermore, the ICR mentions that there was a revision of targets, but it does not clearly state the revisions made and associated rationale.

    Since the implementation of activities may differ between the Bank and other major implementing partners in HSSP the ICR would have benefitted from a discussion and analysis thereof.

    Nevertheless, the ICR does provide substantial relevant information and is thus rated Satisfactory.

    a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions