Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Multisectoral Hiv/aids Project


  
1. Project Data:   
ICR Review Date Posted:
01/09/2013   
Country:
Central African Republic
PROJ ID:
P073525
Appraisal
Actual
Project Name:
Multisectoral Hiv/aids Project
Project Costs(US $M)
 18.0  18.95
L/C Number:
C3584
Loan/Credit (US $M)
 17.0  18.95
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  12/14/2001
 
 
Closing Date
06/30/2006 03/31/2012
Sector(s):
Health (47%), Other social services (33%), Central government administration (15%), Sub-national government administration (5%)
Theme(s):
HIV/AIDS (29% - P) Participation and civic engagement (29% - P) Population and reproductive health (14% - S) Health system performance (14% - S) Gender (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judith Hahn Gaubatz
Judyth L. Twigg Navin Girishankar IEGPS2

2. Project Objectives and Components:

a. Objectives:


    According to the Project Appraisal Document (PAD, page 2) and the Development Credit Agreement (DCA, Schedule 2), the project objectives were as follows:
      • (i) to contribute to reducing HIV/AIDS prevalence; and
      • (ii) to mitigate the impact of HIV/AIDS on persons infected with or affected by HIV/AIDS.

    Emergency needs arising from recent civil conflict, as well as the availability of substantial new resources for HIV/AIDS from the Global Fund, led to a significant shift in project focus. The project was restructured in 2007 and renamed the Multisectoral HIV/AIDS, Health and Education Emergency Support Project. According to the Project Paper (page 2) and the Amendment to the Development Credit Agreement (page 7), the revised objectives were as follows:
      • (i) to contribute to urgent needs in the fight against HIV, such as access to prevention, treatment, and impact mitigation services; and
      • (ii) to contribute to urgent social needs such as prevention of malaria, increased access to water supply, and improved primary education.

    Due to the previously mentioned civil conflict, the Government fell into non-accrual status due to accumulated arrears; therefore, Bank operations were suspended from February 2002 until November 2006. The project under the original objectives, although approved in December 2001, never became effective and did not disburse any funds. Therefore, the project under the revised objectives only is assessed in this review.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 06/25/2007

c. Components:

The components under the original project objectives were as follows:

1. Strengthening Public Sector Responses (Appraisal: US$3.4 million; Actual: US$0): This component aimed to support HIV/AIDS activities of various public sector ministries, with particular focus on the Ministries of Health and Population, Education, Defense, Women and Social Services, and Rural Development. Support to the Ministry of Health was to include strengthening its critical role in developing and coordinating policies and standards, as well as implementing specific interventions such as testing and counseling, care and support for people living with HIV/AIDS (PLWHAs), and treatment for opportunistic infections. Support to non-Health ministries was to emphasize behavioral change (including among women, uniformed personnel, commercial sex workers, truckers, and youth) and care and support activities for PLWHAs.

2. Strengthening Responses by Civil Society Organizations (Appraisal: US$11.4 million; Actual: US$0): This component supported two types of responses from civil society: (i) HIV/AIDS activities of civil society organizations (CSOs) and (ii) community-led grants. For CSOs, the project would support action plans that include prevention, care, and treatment of sexually transmitted infections and opportunistic infections. For communities, the project would provide grants to stimulate HIV/AIDS responses at the community level based on community-led situation analyses.

3. Coordination, Financial Management, and Monitoring and Evaluation (Appraisal: US$2.7 million, Actual: US$0): This component was to support operations of the Technical Secretariat and the Financial Management Unit of the Comite national de lutte contre le SIDA (CNLS), including monitoring and evaluation and audit functions. The HIV/AIDS surveillance system was to be strengthened, and a baseline survey on HIV/AIDS was to be conducted during the first year of the project.

The components under the revised project objectives were as follows:

1. Support to the National Program Against HIV/AIDS (Appraisal: US$6.46 million; Actual: US$5.72 million): This component continued support to civil society and public sector HIV/AIDS responses. However, the following changes were made to the specific activities: Within civil society, the project would focus support on 50 youth organizations, a federation of associations of people living with HIV/AIDS (PLWHAs), and 30 PLWHA associations; within the Ministry of Health (which now had a dedicated project sub-component), the project would focus on geographic areas not covered by the Global Fund or other donors (according to the project team, 31 project sites were selected in rural areas, with the intention of establishing services in remote areas where none existed previously); within non-Health ministries, the project would prioritize the Ministries of Education, Defense, Social and Public Services.

2. Support to Emergency Health Needs (Appraisal: US$6.56 million; Actual: US$8.33 million): This component aimed to support (i) malaria prevention; (ii) access to emergency health services; and (iii) small community-led sub-projects addressing emergency health needs such as rural water supply and drainage of marshlands. This included financing of about 300,000 insecticide-treated bednets for pregnant women and children under five, and equipment for emergency health services.

3. Support to Emergency Education Needs (Appraisal: US$2.7 million; Actual: US$2.64 million): This component aimed to improve the quality of teaching at the primary school level. Specifically, the project would finance teacher training and educational materials for schools in rural areas.

4. Support to a Financial Management and Procurement Unit (Appraisal: US$0.2 million; Actual: US$2.26 million): This component supported establishment of a fiduciary unit to channel funds to the three implementing agencies (CNLS, Health, and Education).

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost:

    • The actual project cost was US$18.95 million, compared to the appraised amount of US$18.0 million. The difference was likely due to exchange rate fluctuations between SDR and US$, as, according to the project team, there were no specific actions that caused the difference.
    • The actual cost for revised Component 4 was US$2.26 million, significantly higher than the appraised cost of US$0.2 million. According to the project team, this was due to the financing of a Project Coordinator position, as well as a technical cell within the project management unit to manage the small community-led sub-projects. This technical cell was initially envisioned to come from a semi-autonomous government unit.

Financing:
    • The actual Bank financing was SDR 13.3 million, which converted to US$18.95 million at project closing. This covered 100% of project costs.

Borrower Contribution:
    • Under the original credit agreement, the Borrower was to provide US$1.0 million in counterpart funding. However, under the amended agreement, the Borrower obligations were eliminated.

Dates:
    • February 2002: The government fell into non-accrual status before the project became effective, and disbursements under the project (and for the entire Bank portfolio) were suspended.
    • June 2006: The project closing date was extended from June 2006 to June 2007 to allow preparations for project restructuring.
    • November 2006: The government cleared its arrears and the Bank lifted the suspension on disbursements.
    • June 2007: The project was formally restructured (with Board approval) to broaden the project scope. The restructured project responded to emergency needs in the social sector that arose from the civil conflict. The project objectives were revised, and the project closing date was extended from June 2007 to September 2010 to allow for implementation of the newly restructured project components.
    • September 2010: The project was revised to refine the results framework (i.e. wording of the indicators) and to extend the project closing date again for a third time from September 2010 to March 2012 to allow for completion of project implementation.
    • November 2011: The project was revised to reallocate credit proceeds among various disbursement categories.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High. The Joint Country Partnership Strategy (2009-2012) and the government's Poverty Reduction Strategy Paper (2011-2015) prioritize human capital development, especially in the areas of health, education, and HIV/AIDS (as the HIV prevalence rate in the country had been estimated at 6.2% in 2006), as well as access to basic social services. These needs were further highlighted by the recent civil conflict, which had led to a deterioration in basic services and in human development indicators for the overall population. The project objectives were also highly relevant to the Bank's corporate priorities as reflected in the Millennium Development Goals.

b. Relevance of Design:

Substantial. The project design with regards to HIV/AIDS activities provided for an integrated approach to HIV/AIDS service provision, for example providing a range of psycho-social, economic and medical treatment services in a single facility in project areas. These areas were selected based on high HIV prevalence levels and were located at the decentralized sub-prefecture level in more remote rural areas. This design was intended to complement Global Fund activities which were more targeted towards the capital city. With the disproportionately higher rate of HIV prevalence among young women, the project emphasized prevention of mother-to-child transmission. A national seroprevalence study conducted in 2006 had identified the highest prevalence rate among uniformed personnel (military and police), at 21.1%, and therefore, the Ministry of National Defense was prioritized for receiving public sector funding. With regards to other activities in the health and education sectors, the project design focused on selected interventions given the limited resource envelope. These included distribution of insecticide-treated bed nets, access to water supply, and provision of materials and teachers for primary education, all of which aimed to cover identified gaps in service provision.


4. Achievement of Objectives (Efficacy) :


During the period of suspension, preparation activities continued to a limited extent with financing from the Project Preparation Facility (US$2.0 million) and Low-Income Countries Under Stress (LICUS) grants (US$0.5 million). These activities included establishment of the HIV/AIDS Technical Secretariat and 16 provincial HIV/AIDS committees, capacity building, pilot activities by selected civil society organizations and communities, and mapping of HIV/AIDS prevalence in at-risk areas.

To contribute to urgent needs in the fight against HIV, such as access to prevention, treatment, and impact mitigation services. Substantial

Outputs
    • Implementation of HIV action plans for the Ministries of Education, Social Affairs, National Defense, and Public Services. Activities were directed towards each of the Ministries' constituents and included information communications, condom distribution, counseling and testing campaigns, psycho-social support, and treatment of opportunistic infections. Of particular note, given the importance of targeting high-risk groups, the project reached a total of 6.845 military personnel and their families in the targeted military camps.
    • Implementation of 85 youth-targeted HIV action plans by 27 youth associations, focusing on behavior change and promotion of counseling and testing.
    • Training of 25 health staff and support groups to conduct promotional work on prevention of mother-to-child transmission. Provision of supplies and HIV tests to support prevention of mother-to-child transmission interventions.
    • Implementation of 35 action plans by PLWHA associations, which primarily focused on income-generation activities.
    • Provision of training and supplies to laboratory staff at project sites for blood safety interventions.
    • Provision of anti-retroviral treatment drugs and supplies.
    • Implementation of medical waste management measures in the 31 project health sites.

Outcomes
    • Voluntary Counseling and Testing (VCT): 118,862 people accessed VCT services in project areas. This fell short of the target of 200,000 people, due to a delay in the delivery of supplies (although the final lot was being distributed at the time of project closing). The total number of people who had accessed VCT services nationwide was 86,149 from 2006-2009.
    • Prevention of Mother-to-Child Transmission (PMTCT): 10,465 pregnant women were tested through PMTCT services in project areas, surpassing the target of 9,000. PMTCT services were expanded from 62 sites to 105 sites nationwide, of which the project sites accounted for 30.
    • Blood Safety: 27,944 blood transfusion pockets were tested for HIV and other blood-borne infections in project areas, falling short of the target of 32,000. The number of pockets in one final project distribution were not available. The project provided almost the sole source of funding for pockets, tests, and consumables for blood transfusions during the project period.
    • Anti-Retroviral Treatment (ART): 1,719 adults and children were receiving ART services in project areas, surpassing the target of 1,300. The number of people receiving ART services nationwide was 14,405. The project particularly provided access to ART services at the sub-prefecture level, as they had only been provided in the capital city and some provincial sites until 2008, mainly through support from the Global Fund.
    • People Living with HIV/AIDS (PLWHAs): 14,004 PLWHAs benefitted from support from PLWHA associations in project areas, surpassing the target of 8,000. As noted previously, the support was primarily focused on income generation. The Global Fund was providing support to orphans and vulnerable children.


To contribute to urgent social needs such as prevention of malaria, increased access to water supply, and improved primary education. Modest, due to shortcomings in malaria prevention.

Outputs
    • Distribution of 100,000 insecticide-treated nets (ITNs), falling far short of the target of 300,000 nets. A second lot of ITNs purchased could not be distributed due to an issue of non-compliance with technical standards of the nets.
    • Provision of sensitization activities and training to 2,400 workers for ITN distribution.
    • Approval of 192 community sub-projects, which included construction and/or repair of 101 community water access points. No sub-project proposals for drainage of marshlands were received.
    • Recruitment and training of 1,023 apprentice teachers in rural areas, surpassing the target of 850. Provision of 88,530 textbooks to primary schools, falling short of the target of 155,560.
    • Provision of emergency medical equipment to 15 health facilities..

Outcomes
    • Malaria prevention: 100,000 households were utilizing ITNs, falling short of the target of 300,000.
    • Access to water supply: 30,300 people were provided access to improved water sources in project areas, surpassing the target of 19,800. The proportion of people with access to drinking water increased from 52% in 2006 to 67% in 2010; however, these represent nationwide figures rather than project-specific areas.
    • Improved primary education: The ICR (page 31) reports that the pupil-teacher ratio declined in project areas; however, no specific data are reported. 4,668 teachers met minimum teaching qualifications in project areas, an increase from the 2007 baseline of 3,666 teachers and surpassing the target of 4,516. The student/books ratio in project areas decreased from 6.7 in 2007 to 1.85 in 2012. There were virtually no schools open in rural areas during the conflict period, and there was no other major donor financing on primary education during the project period.

5. Efficiency:

Modest, due to lack of sufficient information. Neither the PAD nor the Project Paper estimated a net present value or economic rate of return for the project, and none were calculated after project closing.

The project implemented activities under the Ministry of National Defense, whose constituents were targeted as high risk groups for transmitting HIV infection. Other aspects of implementation that indicated efficient use of resources: interventions identified by prior regional work as cost-effective (prevention of mother to child transmission and blood safety); effective coordination mechanism with donors that reduced duplication; and provision of a comprehensive package of HIV/AIDS services at a single site in each project area which may have led to reduced administrative costs. However, no specific data is provided.

For the malaria activities, although distribution of insecticide-treated nets (ITNs) is widely considered one of the most cost-effective malaria interventions, the second lot of ITNs was purchased but not distributed.

For the water supply and education activities, the ICR (page 33) again points to areas of likely efficiency, including repair vs. new construction of water pumps; and provision of school materials and teacher training (which had been identified in a country education report as among the most cost-effective education interventions). However, no specific data is provided.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Moderately Satisfactory. The relevance of the project objectives is rated High, while the relevance of the project design is rated Substantial. Achievement of the objective to contribute to urgent needs in the fight against HIV is rated Substantial, while achievement of the objective to contribute to urgent social needs such as prevention of malaria, increased access to water supply, and improved primary education is rated Modest due to shortcomings in the implementation of malaria activities. Efficiency is rated Modest due to insufficient information.

Note: According to IEG/OPCS Harmonized Evaluation Criteria, when the project objectives are formally revised, the project outcome rating is to be assessed against both the original and revised objectives, with the two ratings weighted according to the share of total disbursements disbursed at the time of the restructuring. However, in this instance, the project under the original objectives did not disburse any funds and therefore its outcome was not assessed.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:


The stability of the political situation improved during the project period, including with the formation of the National Unity Government under the incumbent president in 2009. However, sporadic violence remains an ongoing threat. Interventions for HIV/AIDS will likely be continued (through financing from the Global Fund and the government (in principle, to keep the minimum capacity in place until other funding is made available)), thereby sustaining access to HIV/AIDS services. Interventions in education, water, and general health are also likely to be continued (through follow-up Bank projects in the related sectors).

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

There was a long period of project suspension due to the government arrears situation; however, the Bank team was able to maintain some degree of dialogue and support to the government, including through Project Preparation Funds which were granted on an exceptional basis. US$2.0 million in preparation funds supported activities such as the establishment of the National HIV/AIDS Secretariat and its 16 prefecture-level entities; pilot activities for selected civil society organizations; capacity building; and mapping of HIV/AIDS prevalence in at-risk areas. According to the ICR (page 22), these activities helped to establish a functioning institutional framework that brought in significant support from other donors (for example, testing and treatment support from the Global Fund), and to inform the revised project design according to results of the various pilot activities and studies.

A series of Bank interim strategy notes re-established the Bank's relationship with the country after the period of suspension. The re-engagement approach included a proposal to significantly restructure the HIV/AIDS project to include other social sectors, as it was the only Bank project, out of five, that was focused on the social sector at the time of the Bank's re-engagement in 2006. The Bank team was able to prepare the restructured project within a brief time frame. There were shortcomings in the original M&E framework, specifically the use of HIV prevalence rate as a key indicator. The revised M&E framework saw some improvements, although it would have benefited from more comprehensiveness in the choice of indicators (see Section 10). Risks, including continued political instability and inadequate institutional capacity, were also re-assessed at restructuring, with appropriate mitigation measures identified.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

The initial supervision period was marked by fine-tuning of sector action plans to ensure that the project activities were kept within the overall budget envelope, as well as aligning project procedures. Even though a formal mid-term review was not planned, the Bank team conducted a project evaluation in March 2009 and concluded that achievement of objectives was likely, recommending some budget reallocations and a project extension. The ICR (page 35) cites interviews with authorities who "expressed their satisfaction with the Bank's technical and operational contribution to the project." No major fiduciary problems were reported, with the notable exception of the second lot of bednets purchased that did not conform to required technical standards (see Section 11) and therefore could not be distributed. The ICR (page 35) notes some instances of unrealistic Implementation Supervision Report ratings (i.e. procurement remained Satisfactory even after the issue with the second lot of bednets was made known).

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government fell into arrears shortly after the project was approved, and therefore disbursements were suspended for a multi-year period. The government eventually cleared its arrears and moved forward quickly with project restructuring, including establishing the necessary institutional structures.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:

The implementing agency for the HIV/AIDS activities was the National HIV/AIDS Secretariat, while the Ministries of Health, Planning, and Education oversaw activities in their respective sectors for Components 2 and 3. The The ICR (page 36) cites initial challenges of confusion in the division of responsibilities among the multiple implementing agencies (particularly between the Ministry of Health and the Ministry of Planning on the emergency health-related activities) The performance of the Project Management Unit, after an initial absence of leadership, improved after the arrival of a new Coordinator in 2009 who was effective in addressing implementation challenges as well as coordinating activities among the various sectors. The aforementioned issue with the procurement of bednets led to major shortcomings in the malaria prevention sub-component and remains unresolved (see Section 11). There were no major financial management problems reported.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The M&E framework under the original objectives had a notable shortcoming in that HIV prevalence rate was not an appropriate indicator for measuring project outcomes given the numerous factors that would impact prevalence rates. The framework was refined during project restructuring, and while the revised indicators were overall relevant and measurable, given the critical importance of HIV prevention, additional indicators measuring outcomes in prevention would have been beneficial. According to the project team, as the restructured project was coming after a period of civil unrest, the collection of baseline data was not practical (i.e. many schools had not been functioning and therefore there were no education data to collect). The allocation for M&E activities was significantly reduced at restructuring, given the shift in project focus to an emergency approach, and more structural problems (data consolidation from field to central level, absence of data analysis to impact decision-making processes) related to M&E could not be realistically addressed by the restructured project. However, each sector was allocated a limited amount for M&E functions.

b. M&E Implementation:

The Health Management Information System was operational but unreliable due to poor data quality. M&E implementation improved after the arrival of the PMU coordinator in 2009, which resulted in the introduction of an incentive payment for timely submission of reports; organization of quarterly review meetings that were attended by all sector agencies and various civil society organizations and included reviews of indicator progress and collective validation of results; and revision of the reporting format to ensure consistency across sectors. Each sector had responsibility for data collection and reporting on project outcomes within its targeted areas. Several important studies were completed, including the Multiple Indicator Cluster Survey 4, a second generation sero-surveillance survey, and a health facility survey.

a. M&E Utilization:
According to the ICR (page 24), there was an increased level of interest in using data from existing studies.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The project was classified as an Environmental Category "B" project due to medical waste issues (OP/BP 4.01) and community sub-projects related to drainage of small marshlands and rural water supply (OP/BP 4.04 and OP 4.09). A Medical Waste Management Plan was developed and implemented under the HIV/AIDS component, although "activities were somewhat restricted as a result of a reduced budget for safeguards" (ICR, page 24). According to the project team, overall safeguard compliance was satisfactory with no major issues reported. An Environmental and Social Management Framework was developed, initially under the auspices of the Directorate General of Hydraulics. However, responsibility was shifted in 2010 to the Ministry of Environment, as the Directorate General had limited capacity to oversee matters outside of water supply. The Ministry of Environment was effective in identifying sites for community sub-projects based on environmental screening criteria and sensitizing/training health personnel on medical waste issues. The ICR (page 24) reports that a more timely shift in responsibilities would have lessened implementation delays.

b. Fiduciary Compliance:

Financial management: There were no major difficulties reported in financial management, aside from an initial slow pace of disbursements when the restructured project became effective. There were no issues reported related to ineligible expenditures or any qualified audits.

Procurement: The ICR (page 25) reports that overall, procurement plans were adequately developed and preparation of bidding documents was carried out in a satisfactory manner. One major shortcoming was the purchase of the second lot of bednets, which were not in conformity with required technical standards and therefore could not be distributed. The situation remains unresolved between the supplier and the government, with the nets stocked in a warehouse in Bangui until a decision is taken by the government and appropriate legal measures are taken regarding the supplier. A contributing factor was a general lack of clarity in the accountability mechanism, as the PMU was responsible for processing contracts while technical aspects of the project were the responsibility of each sector focal point.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

Lessons from the ICR, with adaption from IEG:
    • In the context of reduced operations, project preparation funds can be effectively utilized to lay critical groundwork for when operations are resumed in full (page 37). In the case of this project, project advances and trust fund support were utilized to strengthen institutional capacity and helped to leverage funds from the Global Fund and other donors.
    • There need to be clearly-defined responsibilities and accountability mechanisms between the central fiduciary unit and the technical focal points, to ensure that procurements are meeting the appropriate technical requirements (page 37). In the case of this project, while centralization of fiduciary responsibilities helped to reduce transaction costs, the proper technical input was not received in procurement of the bednets.

Lessons from IEG:
    • In the absence of (i) comprehensive baseline data and (ii) a dedicated M&E project component, due to the emergency nature of the situation, individual sectoral agencies can still be made responsible for M&E on a more limited scale to allow assessment of outcomes.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The quality of the ICR is rated Satisfactory. The project outputs were sufficiently detailed to inform the results chain and assess outcomes. The analysis of achievements discussed not only whether output targets were met, but also considered whether there was impact on development outcomes (i.e. increased coverage levels, attribution to the project's activities). The lessons were drawn from project implementation experience. The ICR overall was concisely written and consistent with guidelines.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions