Search in:
Go
Advanced Search

Implementation Completion Report (ICR) Review - Drc Multisectoral Hiv/aids Project


  
1. Project Data:   
ICR Review Date Posted:
04/19/2012   
Country:
Congo Democratic Republic
PROJ ID:
P082516
Appraisal
Actual
Project Name:
Drc Multisectoral Hiv/aids Project
Project Costs(US $M)
 102.0  104.2
L/C Number:
CH080
Loan/Credit (US $M)
 102.0  104.2
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  03/26/2004
 
 
Closing Date
01/31/2011 05/31/2011
Sector(s):
Other social services (45%), Sub-national government administration (32%), Health (23%)
Theme(s):
HIV/AIDS (40% - P) Tuberculosis (20% - S) Social risk mitigation (20% - S) Population and reproductive health (20% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judyth L. Twigg
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
According to the Development Grant Agreement (DGA, p. 22), the objective of the project was “to mitigate the negative impact of the HIV/AIDS epidemic on the development of the Recipient through: (a) reducing the risk of sexual, intravenous, and vertical transmission of HIV; (b) improving the health status and quality of life of people living with HIV and AIDS (PLWHA); and (c) mitigating the socio-economic impact of the epidemic on vulnerable population groups.” The Project Appraisal Document (PAD, p. 2) states the first part of the objective slightly differently, “to mitigate the negative impact of the HIV/AIDS epidemic on the stabilization, recovery, and development of the Democratic Republic of Congo (DRC).”

This Review will assess the achievement of the sub-objectives as stated in parts (a), (b), and (c) in both the DGA and the PAD, as these are the outcomes that were clearly intended to comprise and contribute to the achievement of the overarching objective.

A Level 1 restructuring was approved by the Board on May 29, 2007. It was part of an umbrella restructuring of Multi-Country HIV/AIDS Program (MAP) projects, prompted by recognition that HIV prevalence was unrealistic as a performance indicator, that weak or inefficient institutional arrangements and project design were slowing implementation, and that important lessons had been learned from early project implementation. According to the Project Paper for the umbrella restructuring (p. 9), the revised objectives were “to assist the Recipient in: (i) increasing access to sexually transmitted infection (STI)/HIV/AIDS treatment; (ii) mitigating the health and socio-economic impact of HIV/AIDS at the individual, household, and community level; and (iii) building strong and sustainable national capacity to respond to the HIV/AIDS epidemic.”

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 05/29/2007

c. Components:
The project contained four components:


1. Response of the public sector (appraisal: US$ 40.4 million; actual: US$ 30.8 million, or 76.2% of appraisal). This component was to assist and support the mobilization of public institutions in the fight against HIV/AIDS by mainstreaming HIV/AIDS activities into their regular work programs. Activities by public sector institutions on behalf of their staffs, families, and clients were to be financed on a competitive basis, with funding allocated according to quality of proposals. There were two subcomponents: (a) Program of the Ministry of Health (MOH), which was to consolidate and expand activities in the areas of blood transfusion safety, screening and treatment of sexually-transmitted infections (STIs) and opportunistic infections (OIs), voluntary counseling and testing (VCT), and condom promotion and distribution. This subcomponent was also to play a key role in providing access to anti-retroviral medications (ARVs) and prevention of mother-to-child transmission (PMTCT). Support activities such as epidemiological surveillance and research were also included. (b) Programs of other ministries, which were to provide technical support and finance work plans proposed by other ministries and public sector institutions.

2. Response of the non-governmental sector (appraisal: US$ 17.9 million; actual: US$ 17.1 million, or 95.5% of appraisal). This component was to support HIV/AIDS activities proposed by the HIV/AIDS Inter-Enterprise Committee (CIELS), an umbrella organization for private sector entities and public enterprises that was to coordinate submission of proposals by member enterprises at the national, provincial, and local levels. The component was also to support non-governmental organizations (NGOs) and faith-based organizations (FBOs) through support for work plans, sub-projects, and micro-projects at the provincial and local levels.

3. Support for community initiatives (appraisal: US$ 19.3 million; actual: US$ 5.1 million, or 26.4% of appraisal). This component was to increase the impact of HIV/AIDS activities by transferring responsibility for identifying priorities, and preparing and implementing micro-projects, directly to beneficiaries in rural and urban areas. Resources were to be transferred directly to communities based on simplified financing agreements, and operators were to be contracted to assist communities throughout the process.

4. Coordination and communication, monitoring and evaluation (M&E), and capacity building (appraisal: US$ 22.5 million; actual: US$ 50.6 million, or 225.9% of appraisal). The National Coordination Unit (NCU) was to be responsible for coordinating overall implementation of the program, with decentralization to Provincial Coordination Units (PCUs) and Local Coordination Units (LCUs). This component was to develop a communications strategy targeting audiences at all levels, together with a communication network among the various coordination units. M&E activities and evaluation/impact studies were to be financed by this component, together with transversal capacity-building activities to enhance the ability of actors to prepare, submit, and implement work plans, sub-projects, and micro-projects. Most of these services were to be contracted.

Each of these components was to be implemented through a scheme of vertical organization requiring preparation and presentation of project proposals for approval and financing under one of three unit windows: national, provincial, or local. The required level of presentation for project proposals was to depend on the scope, cost, and coverage of the proposed activity, as well as the capacity of the implementing entity.

The components were not formally revised during implementation.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: At the time of the first, Level 1, restructuring, US$ 32.79 million (31.5%)of the grant had been disbursed.

A second, Level 2 restructuring took place on June 7, 2010. The development objectives were not revised at this restructuring. This restructuring separated the functions of the coordinator of the project implementation unit (PIU) from those of the coordinator of the National Program for the Fight Against HIV/AIDS; reallocated some budget categories into a single expenditure category for the uncommitted proceeds of the grant to facilitate disbursement as the project neared completion; and revised the project’s key indicators and targets to adjust to revised project activities and to address difficulties in data collection. The ICR does not state at precisely what level this restructuring was approved.

Financing: The project was financed through an IDA grant, part of the second “tranche” of the Multi-Country HIV/AIDS Program (MAP) approved in 2003. The project team stated that the project's cost overrun was not financed by the Bank, but is now the responsibility of the Government. There was also US$ 1.7 million in non-eligible expenses.

Earlier financing for HIV/AIDS activities in the DRC had been in the form of a US$ 8.0 million component of the Emergency Early Recovery Credit (P075660, 2001-2005, IDA grant of US$ 50 million) and covered blood transfusion security; voluntary testing, diagnosis, and treatment of STIs; and social marketing of condoms.

Borrower Contribution: There was no planned Borrower contribution.

Dates:

The project’s closing date was extended once, from January 31, 2011 to May 31, 2011. The ICR does not provide the date of approval of this extension or the reason for the extension. The project team explained that the extension, granted in November 2010, was to provide additional time to complete project activities. The project team further added that the estimate of remaining project funds at that time was incorrect, resulting in overspending.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Original objectives: Substantial. In 2003, approximately 3 million people in DRC were estimated to be living with HIV/AIDS, representing just over 5% of the adult population. Provincial variations ranged from 4.5% to 7.5%, with estimates indicating that infection rates could top 20% for some vulnerable groups. The objectives were substantially relevant to the Government’s 1999-2008 multi-sectoral strategy for fighting the epidemic, which was based on the promotion of a multi-sectoral approach and the creation of partnerships among all actors and sectors; the encouragement of a central role for PLWHA; and the mobilization of all stakeholders, including Government agencies, private sector entities, NGOs and civil society organizations, and beneficiary communities throughout the country. The project’s objectives are also substantially relevant to the Bank’s current Country Assistance Strategy (2008-2011), which has as two of its five pillars fighting the spread of HIV/AIDS and promoting community dynamics.

Revised objectives: Modest. The revised objectives essentially eliminate preventing the spread of HIV/AIDS as an objective, making them less relevant to the country context, Government strategy, and Bank strategy.

b. Relevance of Design:
Design under the original objectives: Negligible. The PAD (Annex 3, pp. 1-2) contains a detailed results chain linking the project’s development outcomes to intermediate results and concrete outcome indicators. An equally detailed description of the project’s components explains how the project’s activities would plausibly be expected to contribute to these results. However, particularly for a country affected by “decades of Government mismanagement and recent armed conflict” resulting in a “collapse of the country’s economic, social, and physical infrastructure” (PAD, p. 1), the project was extremely complex. Also, alternative outputs would have been expected to have more impact on the achievement of the desired outcomes. The ICR (p. 20) points out that the preparation team adopted the standard MAP model, rather than adapting that model to specific country conditions. The ICR (p. 11) further suggests that selection of a limited number of targeted provinces might have maximized the project’s impact, rather than implementation country-wide; even with a relatively large project budget, the project may not have contained sufficient resources to reach the entire territory of the country, particularly given the challenge of traveling to some of its remote areas (ICR, p. 12). The PAD (p. 13) cites a social assessment survey that identified women, orphans, and young girls as the groups most vulnerable to the epidemic, primarily because their isolation and weak economic power makes them more likely to engage in risky behavior such as sex work. This study also examined options for enhancing behavior change, including a focus on the influence of customary chiefs and traditional healers. Also, the country’s National Strategic Plan to Combat AIDS focuses on sex workers, female victims of sexual violence, internally displaced persons, workers in mobile jobs (truckers, miners, soldiers), and injection drug users as at-risk groups (ICR, p. 43). Yet the project’s design and activities contained no provisions for focus on, or outreach to, these vulnerable groups, or to the community leaders and healers most likely to influence behavior change. The results framework contained only one indicator related to vulnerable groups.


Design under the revised objectives: Modest. The ICR refers to the first restructuring as a “missed opportunity” (p. 12) to concentrate on most-at-risk populations and/or concentrate the response geographically.

At the first restructuring, the performance indicators were found to be either too ambitious or too narrow in relation to current needs, available project funding, and support from other donors. However, at this restructuring, new indicators were not adopted to match the new objectives. Also, the only indicator related to behavior change among vulnerable groups (sex workers) was dropped.

At the second restructuring, less than a year before closure, the scope of activities under each component was reduced. It was considered that initial project design was too optimistic about how quickly implementation capacity could be built in the multiple agencies involved in the project. The number of targeted ministries was reduced from 27 to 16, and the number of targeted NGOs and communities was reduced as well.


4. Achievement of Objectives (Efficacy) :


During the project period, there were other HIV/AIDS-related investments from other donors: the Global Fund (Round 3, US$ 113.6 million, 2005-2010; Round 7, US$ 38.3 million, 2008-2012; Round 8, US$ 56.4 million, 2010-2011; Round 8, US$ 6.9 million, 2010-2012; Round 8, US$ 8.0 million, 2010-2012) and the United States President's Emergency Plan for AIDS Relief (PEPFAR, US$ 75.1 million, 2007-2009), among others. The ICR does not discuss issues of attribution arising from the activities of these other donors.

The ICR cites project support for 3,017 community-based subprojects (p. 32) but provides no information on the content, implementers, or beneficiaries of these subprojects. The project team added that there is incomplete information on how project resources were spent, particularly for the first 3-4 years of implementation.

ORIGINAL OBJECTIVES

Reduce the risk of sexual, intravenous, and vertical transmission of HIV is rated Modest.

According to the ICR's data sheet (p. iii), across the entire country, the percentage of pregnant women living with HIV who received PMTCT increased from 1% in 2004 to 3.7% in 2010. The project itself treated 310 HIV-positive pregnant women in 2010, representing 7.2% of the total number of pregnant women who were treated in the country that year and 0.26% of the estimated total number of pregnant women with HIV. This was far beneath the 2007 target value of 70%. In numerical terms, the 804 who received treatment by the project over the lifetime of the project did not reach the target value of 1,200.

The number of persons age 15 and older who received counseling and testing for HIV through the project and received their results increased from 1,500 in 2004 to 302,308 in 2010, exceeding the 2010 target of 300,000, but not meeting the 2007 target of 800,000. The project team added that there was a disconnect between the number of people reached through project-funded information campaigns and the availability of HIV testing, leaving many people who may have wanted to be tested unable to do so.

The number of units of blood tested according to national norms was 224,309 in 2010, exceeding the 2010 target of 200,000. The percentage of units of blood tested in compliance with national norms varied: 58% in 2007, 99% in 2008, 92% in 2009, and 70% in 2010. The ICR does not specify the project's contribution to blood testing services.

The number of STI cases treated annually according to national norms was 102,074 in 2010, exceeding the 2010 target of 100,000, but not meeting the 2007 target of 300,000.

27.65 million condoms were distributed during the life of the project, exceeding the 2007/2010 target of 25 million. There is no information provided on how these condoms were distributed, or to whom. It should be noted that an additional 38.4 million condoms were purchased in 2010 for distribution in the coming years, but 25 million of these were destroyed in a warehouse fire.

Improve the health status and quality of life of PLWHA is rated Modest.

The number of PLWHA receiving ARV treatment increased from 2,200 in 2004 to 7,940 in 2010, below the 2010 target of 10,000 and far below the 2007 target of 20,000. The percentage of health zones providing care and distribution of drugs for PLWHA increased from 14% in 2005 to 37% in 2010, not reaching the target of 50%.

Mitigate the socio-economic impact of the epidemic on vulnerable population groups is rated Negligible.

The number of orphans and vulnerable children (OVCs) with schooling provided through the project was 7,815 in 2009, not reaching the 2007/2010 target of 15,000. The project team clarified that the schooling was provided at the primary level.

The percentage of the population with tolerant attitudes toward PLWHA was 9% in 2007 (according to a DHS survey) and 7% in 2009 (according to a Multiple Indicator Cluster Survey), not reaching the target of 70%.

REVISED OBJECTIVES

Increase access to STI/HIV/AIDS treatment is rated Modest, based on data provided above.

Mitigate the health and socio-economic impact of HIV/AIDS at the individual, household, and community level is rated Negligible, based on the limited data provided above.

Build strong and sustainable national capacity to respond to the HIV/AIDS epidemic is rated Modest.

Since 2007, the eight ministries targeted by the project have established units responsible for HIV/AIDS control. Eight additional ministries, as well as the country’s Parliament, have also set up HIV/AIDS units. No information is provided on the activities, capacity, efficiency, or efficacy of these units. The ICR (p. 33) also cites an increase in score, from 27% in 2003 to 62% in 2009, on a “national policy index,” but it does not define this index or explain its relevance to national capacity to respond to the HIV/AIDS epidemic.


5. Efficiency:

Efficiency is rated Negligible. The PAD (p. 11 and Annex 9) presents an economic analysis showing that the project could be expected to abate the number of new HIV infections from 922,000 (expected without the project) to 162,000 (with the project) in 2008. A cost-benefit analysis showed that the project could avert approximately 3.5 million new infections over the project’s five-year lifecycle. In dollar terms, the analysis demonstrated that the project’s contributions would amount to more than US$ 1 billion in benefits through averting productivity losses and costs of care, saving DRC about 2% of GDP annually (and 5% annually by 2030). The Net Present Value was estimated at US$ 1.5 million for the first five years from the inception of the project, growing to approximately US$ 400,000 annually beyond the project’s lifetime.

The ICR does not conduct a formal economic analysis, but it notes that project spending was very high for a result that put only 7,940 patients on ARV treatment and provided PMTCT to only 804 pregnant women. Average costs for these services in other countries are far below the apparent costs for this project (ICR, p. 17). The project team clarified that there is incomplete information on how much of the project’s resources were spent. It is clear, however, that resources were not efficiently focused on prevention of the spread of the epidemic, and not efficiently targeted at the high-risk groups most likely to transmit infection.

The ICR (p. 17) also points out that the project’s broad focus, both thematically and geographically, was inefficient, and that it would have been more efficient to focus on a few critical thematic areas and carefully limited number of provinces.

Allocative efficiency was also weak. The ICR (p. 17) states that the original allocation to the public sector component was insufficient, given the poor state of the country’s public health system in the aftermath of the conflict, and actual spending on this component was even lower. Coordination and communication took a much larger share of actual project resources than could be considered efficient.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Original objectives: Based on Substantial relevance of objectives, Negligible relevance of design, Negligible efficiency, Modest achievement of two of the project’s three objectives, and Negligible achievement of the other objective, the outcome based on the original project objectives is rated Unsatisfactory (corresponding to a rating of 2 on the 6-point scale).


Revised objectives: Based on Modest relevance of objectives, Modest relevance of design, Negligible efficiency, Modest achievement of two of the three objectives, and Negligible achievement of the other objective, the outcome based on the revised project objectives is rated Unsatisfactory (corresponding to a rating of 2 on the 6-point scale).

The overall project outcome is based on the average of these ratings, weighted by the amount of the total loan disbursed before (32.1%) and after (67.9%) restructuring, as follows:

Outcome – original objectives = Unsatisfactory (2) X .321 = .642
Outcome – revised objectives = Unsatisfactory (2) X .679 = 1.358
Total: 2.00, or Unsatisfactory.

The project’s original objectives were substantially relevant to the country’s epidemiological and social situation, the country’s national HIV/AIDS strategy, and the Bank’s country assistance strategy, but the revised objectives were less relevant as they eliminated prevention of the spread of HIV/AIDS as an objective. Project activities did not target the most at-risk groups, a design shortcoming both before and after restructuring. The resources allocated to project activities were unlikely to be sufficient for full realization of project activities and objectives across the entire country, and it is unclear that resources were invested in appropriately prioritized activities. Targets were far from being met for most indicators. Given the investments of other donors, it is unclear to what extent observed outcomes can be attributed to Bank-financed interventions. Efficiency was low, with resources not focused on activities most likely to prevent the spread of the epidemic, few beneficiaries apparently served in relation to project dollars spent, and much more spent than planned on the project’s administrative costs.

a. Outcome Rating: Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

The ICR (p. 19) states that the project helped to consolidate and strengthen the capacities of decentralized bodies (in all provinces and in ten districts) and of hundreds of NGOs, umbrella groups, and communities, but it does not provide detail on these project activities. The presence of the national program has been established in the provinces, but it is not clear that the technical or financial means exist at the provincial level to sustain implementation and monitoring of that program. Financial risk is high, as resources for the fight against HIV/AIDS are provided almost exclusively by external sources. According to the project team, the Bank is closing its office in Kinshasa. According to the Summary of the Borrower’s ICR (ICR, p. 47), the country’s Medium-Term Expenditure Framework has unproductively focused on the gap between available resources and needs, rather than assessing priority areas on which an effective response could be based. High-risk groups are not being reached with targeted prevention activities. The country remains in a conflict situation.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
The preparation process was fully participatory, with key stakeholders in all sectors and at all levels highly involved in project design and in discussing objectives and outcomes. The project was appropriately prepared in partnership with other donors and agencies operating in the country, with support for the establishment of a national framework, headed by the National Multi-Sectoral HIV/AIDS Commission (CNMLS), to coordinate development partners’ activities and resources. The existing contributions of other donors were thoroughly considered (PAD, Annex 2) in order to ensure coordination, lack of duplication, and activity by the Bank in its areas of comparative advantage. The Bank team carefully assessed the capacities of the sectors to be included in the project, and it developed manuals of procedures for the project in general and for the community initiatives component. Lessons learned from previous MAP experience and reflected in the project’s design included the value of a national HIV/AIDS control agency that plays a coordinating rather than implementing role; the importance of contracting of fiduciary management functions to ensure quick and transparent disbursements; the need for leadership and commitment from political authorities; and the optimal role of ministries in making policy, establishing norms and standards, and ensuring quality control, rather then directly executing activities contained in their work plans. A detailed first-year supervision plan was provided in Annex 15 of the PAD; this was highly appropriate, given country conditions and the fast-track preparation process.


However, the ICR (p. 20) notes that the preparatory team adopted and systematically applied the standard MAP country model, rather than adapting it to the particular situation in the country. Only one risk was identified as Substantial (and none were identified as High): the risk that resources would not be sufficient to scale up activities at the national level (PAD, pp. 9-10). Insufficient attention was paid to institutional arrangements at the central level (including with other partners, especially the Global Fund); decentralization modalities, particularly the roles and responsibilities of provincial and district-level bodies; the difficulties of setting up an M&E system; and human resource challenges. A capacity assessment of community-based organizations (CBOs) was not conducted. At the time of the second restructuring, risk identification was much more realistic, with high risk assigned to the likelihood that there would be insufficient government capacity to oversee and manage the project and that lack of coordination between MOH and key partners would limit sustainability of ART treatment. Significant risk was also assigned to the possibility of confusion in the division of responsibilities between the national program coordinator and the project manager, and to delays in the signature of large NGO contracts for community initiatives.

No epidemiological analysis was carried out during preparation to understand better the nature of the HIV epidemic. Baseline data were not collected on key indicators, and as a result, targets for some indicators “were defined based on guesses rather than on knowledge of the true situation of the HIV epidemic” (ICR, p. 8).

A quality enhancement review (QER) of the proposed project, conducted in June 2003, identified three key problems: a loan amount that seemed too large, given the country’s absorptive capacity, previous problems with disbursement, and amounts expected from the Global Fund; the likely overlap of responsibilities between the project’s implementation unit and other structures involved in the fight against HIV/AIDS; and weak technical capacity of the implementing agency. These problems remained addressed during preparation.

Quality-at-Entry Rating: Unsatisfactory

b. Quality of supervision:
There were five different Task Team Leaders (TTLs) during the lifetime of the project. When the project was launched, there was a management decision to base the first TTL in Washington without a dedicated staff person in country. According to the ICR (p. 9), on-the-ground supervision was necessary at the outset of implementation, given the large and diverse nature of the country and its sectoral politics, and the level of required coordination with other development partners. There were only six official supervision missions with Aide Memoires during the seven years of the project, and according to the ICR (p. 21), the composition of the team was not always appropriate. For example, there was apparently an M&E specialist for only one supervision mission. The lack of appropriate personnel meant that inherent problems in implementation were not always detected. Overall, there was insufficient attention given to decentralized supervisory and capacity-building responsibilities. According to the Summary of the Borrower’s ICR (p. 48), the five TTLs all took different approaches to addressing the project’s problems, incorporating points raised in informal exchanges rather than new project elements in the project’s restructurings. As the ICR states (p. 21), the 2007 restructuring did not address the main problems the project was experiencing, and the 2010 restructuring was too late to impact achievement of the project’s objectives. The ICR (p. 21) claims that “it appears that Bank management was not interested in finding lasting solutions to a known problem project.”

The Summary of the Borrower’s ICR (p. 48) further states that Aide Memoires always arrived one or two months late, that the results of implementation status reports (ISRs) were not shared with the project, that the recommendations of the mid-term review (MTR) were not followed, and that communications between the supervision and implementation teams were strained, with long periods of time passing without significant dialogue.

Also, there were significant issues with financial management at the beginning of the project that worsened by the closing date. According to the ICR (p. 21), “legal actions undertaken following these fiduciary problems by some of the implementing agencies are threatening the current portfolio with legal orders to freeze special accounts of Bank-financed operations in the country.” According to the project team, these matters are now being addressed through the country’s legal system.

Quality of Supervision Rating: Highly Unsatisfactory

Overall Bank Performance Rating: Highly Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
According to the Summary of the Borrower’s ICR (p. 49), the Government always supported the fight against HIV/AIDS as a pillar of its poverty reduction strategy, and its commitment was evidenced by its adoption of a national HIV/AIDS strategic plan.

However, the ICR (p. 8) states that the most significant factor affecting project implementation was inadequate management capacity at the central level and extensive leadership turnover. There were five different ministers of health during the lifetime of the project. According to the Project Paper for the first restructuring (p. 10), poor coordination by the MOH during the early years of the project and lack of integration with other health sector programs delayed project implementation. Internal government rivalries, particularly between the Office of the Prime Minister and the Ministry of Health, negatively impacted the ability to change poorly performing project management leadership. Partly because of these rivalries, the multisectoral coordinating body on HIV/AIDS chaired by the Minister of Health met only twice during the project’s lifetime.

The Ministry of Health had weak capacity at the central and provincial levels, and there was inadequate funding of recurrent costs and poor remuneration of staff.

Government Performance Rating: Unsatisfactory

b. Implementing Agency Performance:
There were two implementing agencies: the National Multisectoral Program to Combat AIDS (PNMLS), created by Presidential decree in March 2004, and the financial management agency (FMA) hired for the project. The project experienced high management turnover, with four different project coordinators. Some of these coordinators performed poorly, but they were not removed from their positions due to rivalries among government entities. Two coordinators were in “acting” positions for long periods of time, hampering their ability to take decisive action to improve implementation. In addition, at the time of the first restructuring, the role and responsibilities of the National AIDS Secretariat’s (NAS’s) Project Implementation Unit (PIU) were unclear, resulting in serious delays in decision-making involving NAS, MOH, and the PIU.

Prior to the first restructuring, implementation was critically slowed due to major procurement and financial issues. According to the Project Paper for the restructuring (p. 9), this was caused by the weak performance of the FMA that was recruited by the project to assist in these areas. A Bank-initiated investigation reported that the FMA overlooked financial and procurement mismanagement by both project and MOH staff (these MOH staff were later suspended by the Government). As a result, the Government agreed not to extend the FMA’s contract when it expired in September 2007, and to hire a procurement agent as well as international technical experts in procurement, internal auditing, and financial management. After late 2007, procurement capacity gradually improved.

A supervisory mission in early 2006 revealed that the fiduciary management of the project did not meet minimum standards for transparency and rigor. The situation remained unchanged two years later, with significant but non-reimbursed ineligible expenditures, lack of adequate budgetary management, persistent weaknesses in the control system linked to organizational dysfunction, and poor performance of the FMA. A new FMA hired late in the lifetime of the project resulted in some improvements. However, according to the ICR (p. 10), financial losses of the project are estimated at US$ 7.86 million due to overcommitments and disputed claims.

Implementing Agency Performance Rating: Highly Unsatisfactory

Overall Borrower Performance Rating: Highly Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The PAD (p. 3) and DGA (p. 37) note seven key indicators, with three centered on prevention (% of sex workers using condoms during their last sexual encounter; % of pregnant women with access to PMTCT; % of donated blood tested for HIV and Hepatitis B and C), two centered on quality of life for PLWHA (# of PLWHA receiving quality medical treatment; % of health zones with community services for PLWHA), and two centered on mitigation of impact of the epidemic (% of orphans assisted with tuition and schooling; reduction in discrimination against PLWHA as defined in the Demographic and Health Survey (DHS+)). Targets were set for each indicator, and baselines and data sources were provided. The PAD (p. 8 and Annex 3) also provides a detailed M&E implementation plan, with details of activities and responsibility in the areas of M&E for project implementation, project impact, and evolution and impact of the epidemic. However, the ICR (p. 8) makes it clear that targets were based on guesses rather than on knowledge of the epidemic and its likely trajectory. The project indicators were not fully in accordance with national indicators. The ICR (p. 9) also states that, in practice, coordination responsibility for the overall M&E system was not clear.

At the first restructuring, the key performance indicators were changed to the following: 70% of pregnant women will receive a complete course of ART to prevent mother-to-child transmission; 70% of men and women with advanced HIV infection will receive ART; 800,000 persons age 15 and older will receive counseling and testing for HIV and receive their test results; households containing 15,000 orphans and vulnerable children (OVCs) will receive care and support in the last 12 months; 600 service outlets will carry out blood safety activities; and at least 5,000 subprojects will have received support. There were no indicators for prevention other than those related to PMTCT and blood safety; in particular, there were none related to behavior change among high-risk groups (or any other population).

At the second restructuring, the key performance indicators were changed again to the following: # of pregnant women living with HIV who received PMTCT (target 1,200); # of adults and children with HIV receiving ART (target 10,000, which was less than the 12,595 receiving treatment at the time of the restructuring); # of persons age 15 and older who received counseling and testing for HIV and received their test results (target 300,000); # OVCs with schooling provided through the project (target 15,000, which was less than the 20,183 receiving schooling at the time of the restructuring); # of units of blood collected that are qualified according to national norms (target 200,000); # of condoms distributed (25 million); # of STI cases treated according to national norms (100,000); % of ministries with an operational unit for HIV/AIDS control (60%, which was less than the 100% achieved at the time of restructuring).The Project Paper for the second restructuring (Annex 1) contained a detailed results framework listing indicators, baselines and targets, and data collection frequency, sources, and responsibility for collection and analysis.

b. M&E Implementation:
According to the ICR (p. 9), the adoption of the M&E framework was followed by an action plan, guides for indicators, and training tools and manuals. However, the data collected were not systematically analyzed and processed. Decentralized bodies had insufficient capacity to document data, and there were insufficient qualified technical staff, supplies, and working tools. An M&E specialist was included in only one Bank supervision mission (September 2009), making it difficult to address the poor monitoring arrangements and results framework in a timely manner.

a. M&E Utilization:
The ICR provides no information on the use of M&E data and analysis for policy development or for refinement of project implementation.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The project was categorized “B” and was subject to a partial assessment. It triggered an Environmental Assessment (OP/BP/GP 4.01). Based on the partial assessment, a study was conducted leading to the elaboration of a medical waste management plan. The main provisions of this plan were campaigns to increase awareness, improvement of training to benefit medical and paramedical staff, and additional equipment and adapted technologies for the handling and collection of medical waste in hospitals and health centers. According to the ICR (pp. 9-10), the Bank’s Health Sector Rehabilitation Support Project covers medical waste management related to HIV/AIDS activities, making it unnecessary for this project to duplicate efforts by implementing its own medical waste management plan. The ICR does not state whether this constitutes satisfactory compliance with environmental safeguards. The project team clarified that compliance with environmental safeguards was satisfactory.

b. Fiduciary Compliance:
Prior to the first restructuring, implementation was critically slowed due to major procurement and financial issues. According to the Project Paper for the restructuring (p. 9), this was caused by the weak performance of the financial management agent (FMA) that was recruited by the project to assist in these areas. A Bank-initiated investigation reported that the FMA overlooked financial and procurement mismanagement by the project and by the MOH staff (these staff were later suspended by the Government). As a result, the Government agreed not to extend the FMA’s contract when it expired in September 2007, and to hire a procurement agent as well as international technical experts in procurement, internal auditing, and financial management. After late 2007, procurement capacity gradually improved.

A supervisory mission in early 2006 revealed that the fiduciary management of the project did not meet minimum standards for transparency and rigor. The situation remained unchanged two years later, with significant but non-reimbursed ineligible expenditures, lack of adequate budgetary management, persistent weaknesses in the control system linked to organizational dysfunction, and poor performance of the FMA (Price Waterhouse Coopers, PWC). A new FMA (Klynveld Peat Marwick Goerdeler, KPMG) hired late in the lifetime of the project resulted in improved performance. However, according to the ICR (p. 10), financial losses of the project are estimated at US$ 7.86 million: (i) US$ 2.78 million not recovered from the initial deposit in the designated accounts, (ii) the over- commitment of US$ 2.54 million due to an acceleration of commitments and payments in the last month before the original closing date of the project, (iii) related expenses deemed by the Ministry of Finance not to conform to the original project agreements of US$ 1.78 million, and (iv) disputed claims by the National Multisectoral HIV/AIDS Control Program (PNMLS) of US$ 0.76 million. The project team explained that cost overruns are not being financed by the Bank; the Government has to pay those vendors/providers. Some financial management issues are currently being resolved through the country's court system. The ICR (p. 10) states that the final financial supervision mission found the project in compliance with the obligations of financial reporting and auditing. The project team added that project audit reports were on time and unqualified. It is unclear, however, how audits could have been unqualified, given the ICR's description of the project's fiduciary performance.

c. Unintended Impacts (positive or negative):
None.

d. Other:
None.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Unsatisfactory
Unsatisfactory
 
Risk to Development Outcome:
Significant
High
The country remains in a conflict situation. Financial risk is high, with the fight against HIV/AIDS financed almost exclusively by external sources. It is not clear that the technical or financial means exist at the provincial level to sustain implementation and monitoring of the national HIV/AIDS program.  
Bank Performance:
Unsatisfactory
Highly Unsatisfactory
Supervision missions were unusually infrequent and inadequately staffed, particularly for a known problem project. Many key implementation issues were not addressed in a timely manner, or at all, even during the restructurings. 
Borrower Performance:
Unsatisfactory
Highly Unsatisfactory
There was inadequate management capacity at the central levels of government and extensive leadership turnover. Rivalries among government entities prevented stable and effective project management. There were significant but non-reimbursed ineligible expenditures, lack of adequate budgetary management, persistent weaknesses in the control system linked to organizational dysfunction, and poor performance of the financial management agency. 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons are taken from the ICR, with some adaptation:

Project design in a conflict or post-conflict country must be realistic. In this situation, an in-depth understanding of initial institutional capacity, needs for capacity development, absorptive capacity, and necessary risk mitigation measures should be developed realistically and carefully. In this case, excessive ambition led to an inappropriate plan to cover the entire country with a wide scope of interventions, even though weak institutional systems made achievement of those objectives impossible.

In a post-conflict country, swift and frequent restructuring exercises may be necessary in order to adapt project design and implementation as the situation changes and new data become available. In order for such adaptations to be effective, however, a clear results chain and effective M&E framework is necessary from the outset, to that a causal relationship can be drawn between observed and unobserved results and project activities.

Strict adherence to a global project framework can undermine necessary adjustments to adapt a project to the specific situation in a particular country. Although it is relatively easy to expedite a project under a framework like the MAP, a country’s specific context absolutely must be taken into account during preparation if development objectives are to be achieved.

The absence of a TTL on the ground in post-conflict situations can critically affect a project’s ability to overcome and develop the country’s weak structural systems and make progress toward achievement of development objectives. In this case, the infrequency of supervision missions exacerbated this problem. The early presence of a Bank team on the ground should allow a TTL to identify problems early, be proactive in proposing necessary adjustments and restructurings, exercise intense fiduciary oversight (even when international firms have been hired to provide fiduciary services), and garner necessary management support.


14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is clear, concise, and evidence-based. It provides a thorough explanation for the project’s restructurings, and it accurately follows Bank guidelines for evaluating a project with formally revised objectives. However, it provides very little discussion of the project’s actual activities and outputs that would have plausibly resulted in observed outcomes, particularly with regard to the subprojects. It does not discuss the attribution issues raised by the activities of other donors. Many of the cells in Table 13, “Project Cost by Component,” on page 25 are inaccurate or incomplete. The ICR (p. 11) downgrades the relevance of one of the project’s revised objectives, stating that “at the time of the restructuring it was common knowledge that there is no direct link between treating STI and mitigating HIV.” However, the real issue is that STI treatment should not be the main prevention intervention in a context like this one, and the ICR does not address this issue. Also, the ICR (p. 33) cites a "national policy index," but it does not define this index or explain its relevance to national capacity to respond to the HIV/AIDS epidemic. The ICR does not state whether the project complied with environmental safeguards, and it does not provide information on audits.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)