Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Multisector Hiv/aids Control Project


  
1. Project Data:   
ICR Review Date Posted:
01/09/2013   
Country:
Mauritania
PROJ ID:
P078368
Appraisal
Actual
Project Name:
Multisector Hiv/aids Control Project
Project Costs(US $M)
 23.4  23.0
L/C Number:
CH057
Loan/Credit (US $M)
 21.0  21.7
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  07/07/2003
 
 
Closing Date
03/31/2009 03/31/2012
Sector(s):
Other social services (44%), Central government administration (36%), Health (20%)
Theme(s):
HIV/AIDS (33% - P) Tuberculosis (17% - S) Gender (17% - S) Participation and civic engagement (17% - S) Population and reproductive health (16% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judith Hahn Gaubatz
Judyth L. Twigg Navin Girishankar IEGPS2

2. Project Objectives and Components:

a. Objectives:
According to the Project Appraisal Document (PAD, page 2) and the Development Grant Agreement (DGA, page 18). the objectives of the project were:

  • To maintain the level of HIV infection that causes AIDS below the prevalence rate of 1 percent; and
  • To reduce opportunistic infections.



Under a 2007 umbrella restructuring for several Bank projects in the Africa Multi-Country HIV/AIDS Program (MAP), the project objectives were revised as follows:
  • To increase the coverage and utilization of HIV prevention services, especially for high risk and vulnerable populations; and
  • To increase the coverage and utilization of HIV medical treatment and social care, especially for high risk and vulnerable populations.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 09/05/2007

c. Components:
1. Strengthen Capacity of Government Agencies and Civil Society, and Project Administration (Appraisal: US$6.1 million; Actual: US$5.6 million): This component aimed to establish the national HIV/AIDS institutional framework as well as support project management. Activities included: capacity building for developing national strategies and policies; operational support for the national and regional HIV/AIDS coordinating bodies; capacity building to line ministries, NGOs and community-based organizations to develop and implement HIV/AIDS interventions; advocacy to broaden engagement in HIV/AIDS interventions (including for those reaching high risk groups); and establishment of an M&E system.

2. Expand Public Sector (non-health) Responses for Prevention and Care of HIV/AIDS (Appraisal: US$3.5 million; Actual: US$6.9 million): This component aimed to support HIV/AIDS interventions of (non-health) line ministries. Activities included interventions directed at internal Ministry staff as well as client groups, such as information campaigns, behavior change communications, counseling and testing, and condom distribution. Seven sectoral ministries - Education, Rural Development and Environment, Culture and Islamic Orientation, Women's Affairs, Civil Service, Youth and Sports, Communications, and Uniformed Services - were identified to receive initial funding and were in the process of preparing action plans.

3. Expand Health Sector Responses for Prevention, Treatment and Care of HIV/AIDS, Sexually Transmitted Infections and Tuberculosis (Appraisal: US$4.1 million; Actual: US$3.8 million): This component aimed to support HIV/AIDS interventions from the health sector, given its specialized responsibilities in prevention, treatment and care, as well as its role as a referral unit for other sectors. Activities included: epidemiological and behavioral surveillance; establishment of counseling and testing services; behavioral change communications for safer sex practices; treatment for sexually-transmitted infections; prevention of mother-to-child transmission; blood safety; treatment and psycho-social support to people living with HIV/AIDS; and training of traditional health workers to help reach the population through culturally acceptable channels.

4. Develop and Expand Civil Society Organizations and Private Sector Initiatives (Appraisal: US$7.6 million; Actual: US$6.7 million): This component aimed to build capacity among local communities and in the private sector to implement HIV/AIDS activities. Sub-project grants were to be provided for a broad spectrum of activities, with the aim of reducing high risk behavior and exposure to risk, reducing the vulnerability of those unaware of their risk or unable to protect themselves, and "de-stigmatizing" the disease so as to increase access to services.

The project components were not revised during restructuring; however, several sub-components were dropped (advocacy, training of traditional health workers) or combined (capacity building) in order to simplify the project design.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project cost:

  • The actual project cost was US$23.0 million, slightly lower than the appraised cost of US$23.4 million.

Financing:
  • The actual Grant amount was US$21.7 million, slightly higher than the appraised amount of US$21.0 million. The difference in amounts reflects SDR/US$ exchange rate fluctuations.
  • US$1.48 million of the Grant was cancelled near the end of the project period in order to transfer the funds to another Bank project. It is unclear why the actual amount was higher than the appraised amount, with a significant amount of funds cancelled from the Grant.

Borrower contribution:
  • The estimated Borrower contribution was US$2.4 million. US$1.75 million was provided; however, with the project restructuring in 2007, the remaining counterpart obligation was eliminated.

Dates:
  • October 2007: The project was included in a formal umbrella restructuring of Africa HIV/AIDS projects, which led to revisions of the project objectives and indicators, as well as elimination or consolidation of several project sub-components.
  • August 2008: A military coup took place in the country, which led to the suspension of Bank operations for the following year. During the suspension period, the State General Inspection determined that Bank funds in the amount of US$340,000 were misused and needed to be reimbursed.
  • July 2010: The Government reimbursed the misused funds and requested a retroactive extension of the project closing date.
  • April 2011: The project closing date was retroactively extended from March 2009 to March 2012.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Original objectives
High. Although the HIV prevalence rate in the country was estimated at less than one percent, the country was highly vulnerable to a rapid spread of the epidemic, given the prevailing conditions such as widespread poverty, a mobile population, and certain cultural norms. Therefore, the objective to control the spread of HIV infection was highly relevant. The objective to reduce opportunistic infections was also relevant given the corresponding aim to decrease AIDS morbidity and mortality. The Bank's Country Assistance Strategy for FY08-11 highlights increasing access to basic social services including for HIV/AIDS as a priority. Combating HIV is also a corporate priority for the Bank, as reflected by the Millennium Development Goals. An updated Poverty Reduction Strategy Paper is under preparation, although the most recent Action Plan (2006-2010) identifies combating disease as a key strategic theme.

Revised objectives
High. The revised objectives - similar in nature to the original objectives, although more precise - continue to be highly relevant, for the reasons stated above.

b. Relevance of Design:
Original objectives
Modest. Preventing the spread of HIV infection and addressing its impact was to be achieved through the mobilization of multiple sectors and implementing entities. The specialized role of the health sector was recognized through a separate project component. However, although certain line ministries were prioritized and among the first to prepare action plans, the overall project design did not have a clear and systematic approach for reaching high risk populations (the country's 2002 National Strategic Framework included the following as higher at-risk groups: truck drivers, seasonal rural workers, uniformed personnel, sex workers, fishers and miners), particularly given the lack of reliable data on these groups. This would have been a critical element in containing the epidemic.

Revised objectives
Substantial. The project design under the revised objectives more clearly targeted interventions to high risk groups. According to the project team, this was to be achieved by financing NGOs that worked specifically with the targeted groups (sex workers, migrants, and port workers) as well the largest union of port workers. The Ministry of Defense was expected to include activities with police in its annual action plans.


4. Achievement of Objectives (Efficacy) :

According to the ICR (page 36), the project provided the majority of funding for HIV/AIDS activities in the country from 2004 to2008 (ranging from 66% to 95% of annual funding during that period). However, with the arrival of Global Fund financing in 2009, the share of Bank funding became significantly less, at around 25% from 2009 onwards.

Original objective: To maintain the level of HIV infection below the prevalence rate of 1%
Modest, due to lack of outcomes clearly attributable to the project. The choice of "HIV prevalence rate" as a key indicator was inappropriate given the numerous factors (a number of which are outside the control of the project) that could affect prevalence rates.

Outputs
Public Sector
  • Establishment of 13 sentinel surveillance sites. It appears that some data were collected for specific groups including fishermen, prisoners, commercial sex workers, and patients with sexually-transmitted infections.
  • Training of 23 health sector staff on blood safety.
  • Establishment of 8 blood banks, along with sensitization workshops on blood donation.
  • Support to five treatment centers which provided services to prevent mother-to-child transmission.
  • Ministry of Education: development of HIV/AIDS curriculum modules, distribution of information pamphlets, and establishment of HIV/AIDS youth clubs in schools.
  • Ministry of Defense: establishment of 2 testing centers, provision of supplies for HIV testing, distribution of information pamphlets, and training of paramedical personnel on syndromic management of sexually-transmitted infections.
  • Ministry of Rural Development: distribution of information pamphlets, broadcast of 34 radio spots, and development of HIV/AIDS modules for agricultural extension workers.
  • Ministry of Communication: distribution of information pamphlets, and broadcast of 25 radio spots and 17 television spots.
  • Ministry of Islamic Affairs: development of information guide for Imams, and distribution of information materials to reduce stigmatization.

Civil Society
  • 325 sub-project proposals from NGOs were approved, out of 1,755 submitted. 728 sub-project proposals from community-based organizations were approved, out of 2,820 submitted. There is no information in the ICR on the services provided or the coverage of high risk or vulnerable groups.
  • 23% of the sub-projects approved were from NGOs that targeted high risk groups (specifically, sex workers, migrants, and port workers). However, there is no information on the services provided or the coverage of such groups. In addition, the ICR (page 22) notes that these figures could not be independently verified at the time of the ICR mission.
  • 4 training workshops on HIV awareness and one training workshop on behavior change communication (targeting long distance truck drivers, armed services and youth) for implementing agencies.
  • Inventory of civil society organizations working on HIV/AIDS issues.

Outcomes
  • The HIV prevalence rate was estimated at 0.61% [confidence interval of 0.40-0.82%] among pregnant women aged 15-19 in 2007. It was estimated at 1.45% among uniformed services recruits in 2007 and 4.6% among tuberculosis patients in 2008. according to sentinel surveillance and sector reports. However, attribution of these outcomes to the project's activities is not clear given the numerous factors that would impact prevalence rates.


Original objective: To reduce opportunistic infections
Negligible, due to lack of outcomes reported.

Outputs
  • Provision of treatment to over 17,000 patients with sexually transmitted infections, annually, from 2003-2005.
  • Provision of support (medical, nutritional, psychosocial, or emergency) to 5,518 people living with HIV/AIDS (PLWHAs), surpassing the target of 2,000. However, the ICR (Results Framework Analysis, no pagination) reports that different definitions of "support" and partial data on results make interpretation of the data problematic.

Outcomes
  • There are no outcomes reported on reduction in opportunistic infections.


Revised objective: To increase coverage and utilization of HIV prevention services, especially for high risk and vulnerable populations.
Modest, due to limited evidence of outcomes among high risk and vulnerable populations.

Outputs
(See evidence presented above)

Outcomes
  • Over 1.6 million people were reached by information and education and/or behavior change communications, surpassing the target of 1.2 million. There is no information provided on coverage of high risk groups.
  • Over 5.3 million condoms were distributed over the project period (approximately 0.9 million condoms per year). This fell short of the target of 1.5 million per year; however, the project was the source of almost all condom distribution prior to 2009.
  • 96 pregnant women received treatment to prevent mother-to-child transmission, out of an estimated 500 persons in need.


Revised objective: To increase coverage and utilization of HIV medical treatment and social care, especially for high risk and vulnerable populations.
Modest, due to limited outcomes reported, particularly for high risk and vulnerable populations. The ICR (page 24) also notes that "limited coverage has reduced geographic accessibility and frequent supply chain issues have further reduced the availability and quality of services."

Outputs
  • Distribution of guidelines for voluntary counseling and testing, provision of supplies for testing, and establishment of three testing centers. (Only two testing centers were available prior to 2002.)
  • Provision of anti-retroviral (ARV) therapy at three sites.
  • Provision of support to 670 orphans and/or vulnerable children, out of approximately 3,600 affected children in the country.
  • Support to six PLWHA associations.
  • Passage of legislation protecting PLWHA rights.

Outcomes
  • 1,779 people were receiving anti-retroviral (ARV) treatment, surpassing the target of 1,000. According to the ICR (page 40), the coverage of ARV services ranged from 2.6% in 2004 to 23.3% in 2007. No coverage data are provided after 2007.
  • 89,038 people accessed counseling and testing services and received their test results, surpassing the target of 15,000. Of these, 4,160 people were CEMA (a coalition of private firms) employees, which surpassed the target of 800. The annual number of persons tested increased significantly (from 1,762 in 2005 to 10,930 in 2006) when Bank operations resumed following the project suspension. A mobile testing campaign conducted in 2011 also contributed to exceeding the target.

5. Efficiency:

Modest. The project prioritized key sectors (i.e. through Ministries of Health, Defense) and high risk groups, although the extent to which the latter groups were effectively targeted is unclear. However, there were significant shortcomings in implementation performance that contributed to inefficient use of project resources. These include: inadequate monitoring and evaluation that could have informed decision-making to address efficiency issues; limited coordination with other donors; inadequacies in financial management; and lack of timely payment of counterpart funding.


a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Original objectives
Unsatisfactory. The relevance of the original project objectives is rated High, while relevance of the original project design is rated Modest due to a lack of prioritization of project activities, especially those targeting high risk groups. Achievement of the objective to maintain the HIV prevalence rate below 1% is rated Modest due to lack of sufficient information to verify outcomes. Achievement of the objective to reduce opportunistic infections is rated Negligible due to lack of outcomes reported. Efficiency is rated Modest due to significant shortcomings in implementation performance which would indicate negligible efficiency in the use of project resources.

Revised objectives
Moderately Unsatisfactory. The relevance of the revised project objectives is rated High, while relevance of the project design is rated Substantial. Achievement of the objective to increase coverage and utilization of HIV prevention services is rated Modest due to limited outcomes particularly for high risk and vulnerable populations. Achievement of the objective to increase coverage and utilization of HIV treatment and care services is also rated Modest due to limited outcomes. Efficiency is rated Modest due to significant shortcomings in implementation performance which would indicate negligible efficiency in the use of project resources.

The IEG/OPCS Harmonized Evaluation Criteria stipulate that when the objectives are formally revised, the project outcome rating is to be assessed against the original objectives and the revised objectives, with the two ratings weighted according to the share of total disbursements of the grant at the time of the restructuring. The project under the original objectives had disbursed US$13.2 million out of US$21.0 million, or 62.9%, at the time of restructuring. Therefore, the combined outcome rating is Unsatisfactory.

a. Outcome Rating: Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

Although the socio-cultural environment, particularly amongst the religious community, has improved somewhat, the ICR (page 20) notes that certain key activities, namely behavior change which is a critical element of HIV prevention, "seem to have stagnated... or at least lost a sense of dynamism." A continued risk of inadequate funding, due to a heavy reliance on donor funding, may also affect project impact.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
Due to the lack of comprehensive data (including baseline information) on HIV/AIDS in the country, an extensive range of analytic work was planned during project preparation. This included a rapid assessment of targeted high risk groups, an assessment of the role of the Ministry of Health, a social assessment of potential beneficiaries including migrant men and sex workers, behavioral surveys, and community-based pilot projects. However, as noted in the ICR (page 13), the background analysis "commenced at the same time that the project was being appraised, with the result that certain information was missing..." such as reliable data by region and by target group and mapping of ongoing donor activities. The project components had numerous sub-components, which required a high degree of coordination among many implementing entities. There were shortcomings in the original M&E framework, particularly in the formulation of key indicators (See section 10).

Quality-at-Entry Rating: Unsatisfactory

b. Quality of supervision:
The project was restructured in 2007, as part of an umbrella restructuring of 8 HIV/AIDS projects in the region, with significant revisions to the objective statement, the results framework, and the institutional arrangements. The project components were simplified (the number of sub-components was reduced from 16 to 9, with certain activities eliminated or consolidated) and targeting of high risk and vulnerable groups was improved. The project was more adaptable upon the arrival of substantial new funding from the Global Fund in 2008. However, there was frequent turnover in task team leadership (six different team leaders during the supervision period). The Bank team consistently raised concerns about M&E problems (such as the quality of the information system and the resulting data) and although the key indicators in the M&E framework were revised, the key activity of monitoring high risk groups was not carried out. The Bank team also made efforts to move forward on environmental issues (including through a 100-day action plan in 2006, monthly video conferences, and a study tour in 2007); however very little was accomplished on medical waste management.

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government demonstrated commitment to addressing HIV/AIDS (reflected in part by a Senate discussion on HIV/AIDS which was aired live in August 2002), and adopted measures to facilitate a broad multisectoral response (including legislation, implementation arrangements, and technical strategies). Initial tensions between the National HIV/AIDS Secretariat and the Ministry of Health over institutional authority on HIV/AIDS issues were eventually resolved when the Minister of Health was appointed as the Vice President of the National HIV/AIDS Committee. However, the political environment was unstable through much of the project period due to military coups in 2005 and 2008. Counterpart financing, particularly for regional activities, was often delayed, leading to the elimination of 9 out of 13 regional HIV/AIDS Secretariats due to lack of funding.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
Although the primary implementing agency, the National HIV/AIDS Secretariat, was able to implement a number of project activities, there were significant shortcomings in performance overall. Inadequacies in the M&E system were not fully addressed. There were shortcomings in financial management including late and incomplete reports and qualified audits, and a government-led investigation into misuse of Global Fund monies and ineligible IDA expenditures. There were also shortcomings in safeguard compliance, due to limited implementation of medical waste management activities. The Ministry of Health also had shortcomings in implementing performance with lower than planned coverage of services and failure to establish an effective monitoring system.

Implementing Agency Performance Rating: Unsatisfactory

Overall Borrower Performance Rating: Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The original key indicators, including maintaining of the current HIV prevalence rate, were unrealistic and/or difficult to measure, given the relatively short time frame of the project and the multiple factors (a number of which would be outside the control of the project) that would impact prevalence rates. Although the M&E design appropriately emphasized monitoring the epidemic and and its impact (including an extensive list of surveys to be conducted: social impact surveys, targeted surveys of vulnerable groups, and national surveillance surveys), as well as simplicity in reporting requirements and procedures so that all implementing entities could contribute, capacity to carry out the M&E activities was inadequate. An external agency was to be contracted for M&E, but this did not happen.

b. M&E Implementation:
The key indicators were revised during the project restructuring; however, the key activity of monitoring high risk groups (surveillance and behavioral) was not carried out. The ICR also (page 17) reports that "baseline data, targets, and annual results were not systematically presented by the project but pieced together by the ICR mission from various annual reports [and other donor documents]." Beneficiary assessments were not carried out as planned.

a. M&E Utilization:
There was limited cooperation between the National HIV/AIDS Secretariat and the Ministry of Health, which limited the exchange and utilization of M&E results. There was also a lack of effective mechanisms to exchange and disseminate results.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The project was identified as a Category "B" project due to medical waste management issues, which triggered OP4.01/BP 4.01 on Environment Assessment. A medical waste management plan was prepared and disclosed. However, despite various attempts by the Bank team to move forward on environmental issues, only the training of trainers and distribution of a guide for managing biomedical waste was accomplished. Therefore, safeguard compliance is considered unsatisfactory.

b. Fiduciary Compliance:
Financial management: There were significant shortcomings in financial management performance. A number of annual audits were qualified, citing inadequate controls for the allocation and use of funds and inconsistent accounting results. One Bank audit showed that project funds of about US$340,000 were misused (for example to pay salaries of the National HIV/AIDS Secretariat staff). The Government eventually reimbursed the funds, which was a condition for extension of the project closing date. Financial management at the regional level was especially problematic, with problems in internal controls and eligible expenditures, and lack of rigor in accounting procedures.

Procurement: Despite detailed procurement arrangements in the PAD, initial procurement performance was inadequate, including problems with tenders and inadequate capacity in the procurement unit. However, particularly with the recruitment of additional procurement expertise, performance improved and subsequent supervision missions did not identify significant problems other than some delays in implementing the procurement plan and some shortcomings in the rigor of the review process.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Unsatisfactory
The project under the original objectives is rated Unsatisfactory, as relevance of the project design is rated Modest due to a lack of prioritization of project activities, especially those targeting high risk groups; achievement of the objectives to maintain the HIV prevalence rate below 1% and to reduce opportunistic infections are rated Modest due to lack of sufficient information to verify outcomes; and efficiency is rated Modest. The project under the revised objectives is rated Moderately Unsatisfactory, as achievement of the objectives to increase coverage and utilization of HIV prevention services, as well as HIV treatment and care services are rated Modest due to limited outcomes; efficiency is also rated Modest. The combined outcome is Unsatisfactory, due to a weighted rating by which the original project had disbursed US$13.2 million out of US$21.0 million, or 62.9%, at the time of restructuring.  
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Moderately Unsatisfactory
Unsatisfactory
At entry, there were significant shortcomings such as inadequate baseline understanding of the HIV epidemic in the country, complex implementation arrangements, and inadequate M&E arrangements. 
Borrower Performance:
Moderately Unsatisfactory
Unsatisfactory
The National HIV/AIDS Secretariat and Ministry of Health had significant shortcomings in implementation performance, along with overall inadequate financial management performance. 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Lessons from the ICR, adapted by IEG:
  • A project that is intended to take a "learning by doing" approach should avoid overly cumbersome project design (page 31). In the case of this project, there were numerous project sub-components to be implemented by multiple implementing entities, with limited communication and coordination that did not enable effective learning.
  • A project that is intended to take a "learning by doing" approach should also propose a realistic M&E system (page 32). In the case of this project, the M&E design included an extensive list of activities despite limited capacity, and mechanisms to ensure "learning" were not clearly delineated.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR was overall satisfactory, with regards to conciseness and consistency with guidelines. The quality of evidence (specifically, project outputs) was adequate, although a clearer presentation (see Annex 2a) would have been helpful. Although the analysis was appropriately outcome-driven, data on outcomes among high risk groups were insufficient clearly to demonstrate project impact.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions