Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Emergency Multi-sector Hiv/aids Project


  
1. Project Data:   
ICR Review Date Posted:
02/03/2014   
Country:
Cote d'Ivoire
PROJ ID:
P071631
Appraisal
Actual
Project Name:
Emergency Multi-sector Hiv/aids Project
Project Costs(US $M)
 20.0  20.0
L/C Number:
CH399
Loan/Credit (US $M)
 20.0  20.0
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  06/12/2008
 
 
Closing Date
09/30/2012 09/30/2012
Sector(s):
Health (38%), Central government administration (24%), Other social services (21%), Sub-national government administration (12%), Solid waste management (5%)
Theme(s):
HIV/AIDS (33% - P) Population and reproductive health (17% - S) Other social development (17% - S) Participation and civic engagement (17% - S) Child health (16% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judith Hahn Gaubatz
Judyth L. Twigg Lourdes N. Pagaran IEGPS2

2. Project Objectives and Components:

a. Objectives:


    According to the Financing Agreement (page 5) and the Emergency Project Paper (page 7), the objectives of the project were to support the Borrower’s response to the HIV/AIDS pandemic, primarily in four regions, by:
      • Strengthening access to and increasing utilization of prevention services among vulnerable and high risk groups such as women, youth, commercial sex workers, and personnel of line ministries; and
      • Improving access to and utilization of treatment and care services for HIV/AIDS infected and affected persons, notably persons living with HIV/AIDS, orphans, and vulnerable children.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1. Social Mobilization and HIV: Prevention Services for Vulnerable and High-Risk Groups (Appraisal: US$6.4 million; Actual: US$5.3 million): This component aimed to support NGO- and private sector-led activities for HIV/AIDS prevention, care and treatment, primarily in the four project regions. Activities in the four project regions included: prevention interventions among high risk groups (especially commercial sex workers); care for orphans and vulnerable children; treatment and care for people living with HIV/AIDS; and information and education campaigns, counseling, and testing services for mobile populations, youth, and women living in low-income urban areas. Activities nationwide included information and education campaigns for commercial sex workers, youth, and the general public.

2. Public Sector Interventions (Appraisal: US$7.8 million; Actual: US$7.6 million): This component aimed to support the various public sector ministries in delivering HIV/AIDS prevention, care and treatment. For the Ministry of Health, activities included: anti-retroviral treatment; treatment for opportunistic infections, tuberculosis, and sexually transmitted infections; strengthening of laboratory capacity; prevention of mother-to-child transmission; and epidemiological surveillance. For the Ministries of Defense, Interior, and Education, activities included: information and education campaigns; counseling and testing; condom distribution; HIV/AIDS modules for school curricula; and support for people living with HIV/AIDS.

3. Capacity Building (Appraisal: US$3.3 million; Actual: US$4.7 million): This component aimed to support capacity building to ensure effective implementation of project activities. Activities included training of sector staff, support for sector action plans, and technical assistance to NGOs.

4. Coordination, Management, Monitoring and Evaluation (Appraisal: US$2.5 million; Actual: US$2.4 million): This component aimed to strengthen the national coordination mechanism, the Ministry to Fight HIV/AIDS, as well as regional coordinating bodies. Activities included operationalization of regional bodies, strengthening of surveillance systems, and impact evaluations.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project cost

    • The actual project cost was US$20.0 million, out of the appraised US$20.0 million.

Financing
    • The project was financed by an IDA grant of US$20.0 million, which disbursed in its entirety.

Borrower contribution
    • There was no planned Borrower contribution.

Dates
    • Despite a lengthy period from project concept to appraisal (2002 to 2008) due to political conflict, the project was implemented in the planned period from 2008 to 2012 and closed on schedule.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High. The project objectives were highly relevant to country conditions. The HIV prevalence rate is among the highest in West Africa at 3.7% (2012). In addition, the recent socio-economic and political crisis negatively impacted the availability of basic social services in the country, including for HIV/AIDS prevention and care. The country's National HIV/AIDS Strategic Plan for 2006-2010 underscored the need to reach targeted groups as well as the provision of a comprehensive package of HIV/AIDS services. The objectives were also highly consistent with the priorities identified in the Bank's Country Partnership Strategy (FY10-13), including improved access to basic services such as HIV/AIDS prevention and care.

b. Relevance of Design:

Substantial. The project's design focused on sub-populations at highest risk of transmitting HIV, particularly commercial sex workers. The project's planned interventions were technically sound and widely considered cost-effective, and they were to be implemented largely by civil society organizations and NGOs who were more likely to reach the targeted groups. The project focused on four (out of a total of nineteen) regions of the country, selected according to the level of HIV prevalence, existing level of support from other donors, and ethnic composition. Public sector support included prioritized support to the Ministries of Education, Defense, and the Interior, in order to reach high risk and vulnerable populations. The project's design also provided intensive support to the health sector, which was the primary source of care and treatment services for people infected or affected by HIV/AIDS.


4. Achievement of Objectives (Efficacy) :


To strengthen access to and increase utilization of prevention services among vulnerable and high risk groups:
Achievement of this objective is rated Modest, due to lack of clear evidence of outcomes among high risk groups, who are the main drivers of the epidemic and critical to prevention efforts.

Outputs
Project regions
    • Distribution of over 3.0 million condoms, including to commercial sex workers, men having sex with men, mobile populations, and defense forces, in the four project regions. This fell short of the target of 5.18 million. The ICR (page 23) suggests that the shortfall was due to the interruption in services and supply chain stemming from the crisis.
    • Conducting of information/education/communication (IEC) and behavior change campaigns targeting commercial sex workers, men having sex with men, mobile populations, defense forces, and youth.
    • Provision of integrated HIV services through 59 health facilities, including voluntary counseling and testing, prevention of mother-to-child transmission, treatment for sexually transmitted infections, and anti-retroviral treatment.
    • Provision of integrated HIV services through the army and police health services.
    • Establishment of 7 regional laboratories.
    • Awarding of 81 sub-contracts to NGOs to implement interventions among high risk groups. This surpassed the target of 51.
    • Introduction of HIV/AIDS modules for school curricula.
    • Establishment of AIDS clubs in 86 schools.

Nationwide
    • Conducting of IEC campaigns among the military population to improve knowledge about HIV/AIDS and reduce stigma.
    • Implementation of prevention action plans in 48 small and medium-sized private enterprises.

Outcomes
High risk groups
    • The percentage of female sex workers in the project regions reporting the use of a condom with their most recent client was 93% in 2011. This was slightly below the baseline figure of 95% in 2007 (drawn from a different survey and representing nationwide data) as well as the target of 96%. There were no data reported to provide a more direct comparison before and after project intervention. However, the ICR (page vii) suggests that 93% was a substantial achievement given the following circumstances that arose during the project period: the political-social unrest in 2010-2011 that led to increases in risky behavior; and the number of outlets providing condoms decreasing from 2500 in 2009 to 1500 in 2011.
    • 15,437 commercial sex workers were reached by information campaigns and condom distribution. There is no information on the proportion of total CSWs in the project regions represented by this figure.
    • 12,106 female sex workers received an HIV test in the last 12 months and returned to receive their results. This surpassed the target of 10,500.

Vulnerable groups and/or general population
    • The percentage of women aged 15-49 having had more than one sexual partner in the prior 12 months and reporting condom use increased from 12.0% in 2005 to 29.7% in 2012, surpassing the target of 14.25%. The percentage among men also increased from 30.0% in 2005 to 35.7% in 2012, nearly achieving the target of 36.0%. Within the four project regions, the percentage among both men and women was 37.8% in 2012, compared to 32.2% in non-project regions.
    • 201,647 men and women received an HIV test in the last 12 months and returned to receive their results. This surpassed the target of 100,000. Data comparing project vs. non-project regions showed higher rates of testing in project regions vs. non-project regions (38.6% vs. 30.7%) in 2012.
    • The percentage of females who correctly identify ways to prevent sexual transmission of HIV was 53.1% in 2012 in the four project regions, compared to 47.2% in non-project regions.
    • The percentage of females in low-income urban areas who both correctly identify ways to prevent sexual transmission of HIV and who reject major misconceptions about HIV transmission was 28.0% in 2011 in the four project regions. The percentage of males was 44.0%. The baseline figure of 16.0% in 2005 reflects nationwide results. Therefore, a direct comparison is not possible, but the results suggest the possibility of positive impact on knowledge.
    • 893 pregnant women, which represented 44% of those testing positive for HIV, received a full course of antiretroviral treatment. This fell short of the target of 75%.

To improve access to and utilization of treatment and care services for HIV/AIDS infected and affected persons:
Achievement of this objective is rated Substantial.

Outputs
    • Provision of integrated HIV services through 59 health facilities, including voluntary counseling and testing, treatment for sexually transmitted infections , and anti-retroviral treatment.
    • Training of 1,145 health workers.
    • Support to four regional HIV/AIDS centers to serve as platforms for NGOs supporting orphans and vulnerable children.

Outcomes
    • 51,000 pregnant women were tested for HIV.
    • 12,341 persons living with HIV/AIDS received medical treatment and care. Of those, 2,710 persons received anti-retroviral treatment, surpassing the target of 2,500.

5. Efficiency:

Substantial. The project's prevention interventions centered largely on those that are widely considered cost-effective, including voluntary counseling and testing, condom distribution, and peer education focused on high-risk groups. The project also used the "finger prick" testing method, which cost about US$4.17 per test compared to US$17.73 per test using the traditional method (which requires laboratories). The ICR (Annex 3) provides an estimated benefit-cost ratio of 1.85. Estimated benefits were based on averted morbidity and mortality, resulting from project interventions including condom use by commercial sex workers, condom use among those having multiple sexual partners, prevention of mother-child transmission, anti-retroviral therapy, and HIV testing.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Relevance of the project objectives is rated High, while relevance of the project design is rated Substantial. Achievement of the objective to improve access to and utilization of treatment and care services for HIV/AIDS infected and affected persons is rated Substantial. However, achievement of the objective to strengthen access to and increase utilization of prevention services among vulnerable and high risk groups is rated Modest due to lack of clear evidence of outcomes among high risk groups, who are the main drivers of the epidemic and critical to prevention efforts. Efficiency is rated Substantial.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:


The political and security situation remains precarious but has increasingly stabilized recently. The government remains committed to addressing HIV/AIDS, as evidenced by the development of an updated National HIV/AIDS Strategic Plan for 2011-2015 and securing of substantial funding for the Plan from donors. A National Fund for HIV/AIDS has also been established to finance care and treatment in the longer term, through taxes on alcohol and tobacco.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

Project preparation initially began in November 2000 but experienced multiple disruptions due to political conflict in the country in 2002 and 2004. The project was re-launched in 2008, with significant revisions based on updated epidemiological data and availability of additional resources from other donors. The launch of the project amidst very challenging political and security conditions is notable. The resulting project design focused on four underserved regions (vs. nationwide), targeted prevention activities among high risk groups (especially commercial sex workers), and monitoring and evaluation. The risk assessment appropriately identified the following "High" risks: political instability and insecurity; transitions due to elections; weak implementation and coordination capacity; corruption; and shortages of drugs and medical supplies. Effective mitigation measures were identified, largely through utilizing independent entities to implement and manage the project; however, the risk of drug shortages was not effectively mitigated due to disruptions in the supply chain. The M&E design was focused on the appropriate indicators in terms of measuring impact among key sub-populations; however, neither the baseline and target values, nor the data collection arrangements, were sufficiently revised to reflect the shift from national coverage to four regions only.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

The project continued to experience significant interruptions during the project period, due to political conflict and transitions. These included division of the country into two zones (government vs. rebel controlled), presidential elections in 2010, and armed conflict in 2011. Implementation was particularly affected for the health sector activities, due to disruption in the health system in the two regions formerly controlled by rebels, lack of human resources, and lack of leadership at the sub-national level (ICR, page 9). However, the task team was able to provide regular supervision support, largely enabled by its field presence, and according to the ICR (page 11), more than 85% of planned project activities were executed and the grant was fully disbursed by the original project closing date. Safeguards and fiduciary performance were satisfactory, although M&E shortcomings were not sufficiently addressed.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The government continued to demonstrate strong commitment to the project's objectives, as reflected by overall institutional arrangements and strategic documents in support of addressing HIV/AIDS. This commitment was largely sustained despite the significant security challenges. The presidential elections in 2010 and ensuing political transitions led to a halt in project implementation for a period of nine months as well as a delay in conducting the mid-term review. The government has also shown commitment through its efforts to mobilize financial resources outside of Bank project support.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

The Ministry to Fight HIV/AIDS was responsible for the overall national response, although the political transitions during the project period were a major factor in the lack of regular meetings of the leadership council. The project implementation unit was consistently staffed through the project period and carried out its functions satisfactorily, despite the volatile environment. There were no problems reported in safeguards compliance or fiduciary management. However, there were shortcomings in M&E that affected the quality and availability of data to assess outcomes.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The M&E design drew directly from the country's National HIV/AIDS Strategic Plan, including the indicators and data collection arrangements, to ensure harmonization of HIV/AIDS activities in the country. Baseline data and targets were identified, although these subsequently proved to be inadequate due to the shift from national coverage to four regions only, as well as differing data sources.

b. M&E Implementation:

Routine project data were monitored, and the planned surveillance, impact, and knowledge studies were conducted with support from the project. However, although important surveillance information was collected during the project period, there were difficulties in data comparability due to differing sources of information or geographic scope.

a. M&E Utilization:

As noted in the ICR (page 13), data informing the project outcome indicators were not always available in a timely manner due to timing of the surveys.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The project was classified as a Category "B" project, with the primary environmental issue identified as medical waste management. An existing Medical Waste Management Plan was updated, and the project supported its implementation in project regions through training of health personnel, installation of incinerators, and provision of equipment. Implementation was regularly monitored during supervision missions. The project team confirmed satisfactory safeguards compliance.

b. Fiduciary Compliance:

Financial management: At appraisal, financial management risk was assessed as High; therefore, mitigation measures were identified, such as contracting out fiduciary management to an independent agency and conducting regular independent audits. The use of multiple implementing partners also led to the hiring of an internal auditor to strengthen the financial management system. These measures were implemented effectively, and financial management was overall satisfactory. Audit reports were timely and unqualified.

Procurement: There were no major problems reported in procurement, and overall performance is considered satisfactory.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

Lessons from the ICR, adapted by IEG:
    • Strong political commitment at the highest levels, along with continued engagement of development partners, can help to sustain attention to fighting HIV/AIDS even in the midst of a volatile political and security environment. Flexibility is needed to adjust implementation strategies, in light of changes brought on by the volatility.
    • M&E arrangements need to take into account national vs. sub-national coverage of a project, and adjustments need to be made to baseline and target values, as well as data collection sources, to ensure proper comparability of data.

14. Assessment Recommended?

Yes
Why?
To verify outcomes, particularly in prevention among high risk groups.

15. Comments on Quality of ICR:

The ICR is results-driven and provides useful information on the country conditions, in order to better understand the project context. Lessons are well-drawn from the project's implementation experience. However, the shortcomings in the project's M&E (lack of comparability of data) ultimately limited the quality of the evidence.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions