Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Great Lakes Initiative On Hiv/aids (glia) Support


  
1. Project Data:   
ICR Review Date Posted:
03/26/2012   
Country:
Africa
PROJ ID:
P080413
Appraisal
Actual
Project Name:
Great Lakes Initiative On Hiv/aids (glia) Support
Project Costs(US $M)
 20.0  20.8
L/C Number:
CH150
Loan/Credit (US $M)
 20.0  20.8
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  03/15/2005
 
 
Closing Date
03/31/2009 12/31/2010
Sector(s):
Other social services (60%), Central government administration (40%)
Theme(s):
HIV/AIDS (33% - P) Tuberculosis (17% - S) Health system performance (17% - S) Gender (17% - S) Population and reproductive health (16% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judyth L. Twigg
George T. K. Pitman Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
According to the Development Grant Agreement (DGA, p. 20) and the Project Appraisal Document (PAD, p. 4), the objectives of the project were to support the Recipient in the: (i) establishment of HIV/AIDS prevention, care, treatment, and mitigation programs for mobile and vulnerable groups such as refugees, transport sector workers, and infected/affected populations in the territory of each of the Member States of the Recipient; and (ii) enhancement of prospects for coordinated approaches to HIV/AIDS prevention, care, treatment, and mitigation amongst the Member States of the Recipient.

Member countries of the Recipient, the Great Lakes Initiative on AIDS (GLIA), include Burundi, Democratic Republic of Congo (DRC), Kenya, Rwanda, Tanzania, and Uganda.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project had four components:

1. HIV/AIDS support to refugees, affected areas surrounding refugee communities, internally displaced people (IDPs), and returnees (appraisal, US$ 8.0 million; actual, US$ 8.2 million). This component aimed to provide services to a limited number of target populations, and was intended to include a full range of prevention, care, treatment, and mitigation services. The United Nations High Commissioner for Refugees (UNHCR) was to be responsible for scaling up HIV activities in refugee camps (US$ 5 million), and the National AIDS Commissions (NACs) from the individual countries were to coordinate activities in the surrounding communities (US$ 3.0 million), with each country selecting two priority catchment areas.

2. Support to HIV/AIDS-related networks (appraisal, US$ 3.0 million; actual, US$ 3.6 million). This component aimed to enhance “AIDS Competence” of long-haul transportation workers and people living with HIV and AIDS (PLWHA) by developing: (i) management capacity of specific networks and lead member organizations in planning, financial management, resource mobilization, and monitoring and evaluation (M&E); and (ii) HIV/AIDS learning capacity and transfer of knowledge between network members. Transport networking was to focus on two principal transmission corridors: (i) Mombasa-Nairobi-Kampala-Kigali-Bujumbura-Bukavu-Goma, and (ii) Dar es Salaam-Dodoma-Kigali-Bujumbura-Bukavu-Goma, covering both the truckers and the communities and groups with whom they interact. Sub-regional networking of PLWHA communities and groups would: (i) provide reinforcement of national and under-funded regional advocacy efforts; and (ii) engage in sharing of good practices. Intermediary institutions were to provide management training, and support AIDS competence activities, including pilot knowledge rooms along the transport corridors.

3. Support to regional health sector collaboration (appraisal: US$ 3.0 million; actual, US$ 3.3 million). This component provided support for five key activities: (i) an inventory of effective interventions and sharing of information; (ii) a review of protocols, materials, and training opportunities for prevention and treatment; (iii) information exchange on refugee, IDP, and returnee HIV/AIDS-related health programs; (iv) transport sector HIV/AIDS strategy coordination and piloting of targeted transport packages along two main regional corridors; and (v) information exchange on drug policies and procurement.

4. Project coordination (appraisal: US$ 6.0 million; actual: US$ 5.7 million). This component covered three activities aimed at strengthening GLIA’s capacities: (i) administration and management, including support for core professional staff; (ii) capacity strengthening and policy/technical support including fiduciary, management, and advocacy training, and policy discussion, development, and technical support for other cross-cutting issues (such as gender-sensitive HIV/AIDS services for mobile population programs); and (iii) M&E and reporting.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: Although the ICR does not state this explicitly, it is likely that the difference between estimated and actual project costs is due to fluctuations in the US dollar/XDR exchange rate. The project team confirmed that this was the case. The ICR also does not explain why project coordination costs amounted to such a large percentage of the Loan. The project team explained that coordination and administration costs were relatively high because the project was, in essence, building an institution from the ground up, with an international headquarters and a significant presence in each of the six member countries; also, GLIA headquarters staff was recruited internationally rather than locally, through a six-country selection process.

Borrower Contribution: The project was financed by an International Development Association (IDA) grant. There was no planned Recipient contribution.

Dates: The Mid-Term Review (MTR), conducted in December 2008, recommended a restructuring to formalize proposed changes to the results framework, but this was not done due to the inability of the task team to process the restructuring quickly, difficulties in reaching consensus expeditiously among all six countries, and insufficient Bank management support (ICR, p. 11). The MTR recommendation to extend the time frame for implementation was taken into account with a 21-month extension of the closing date, from March 2009 to December 2010.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High. At the time of appraisal, the cumulative number of refugees and displaced persons in the Great Lakes countries was estimated to be over 6.5 million. In addition, these countries were estimated at appraisal to have over six million PLWHA, with adult (age 15-49) infection rates ranging from 4% to 9%. Even though HIV adult infection rates have since been revised downward in most countries, the estimated overall burden of disease remains high. In light of high levels of mobility and vulnerability, the regional approach embodied in the project’s objectives provided important complementary support to ongoing country HIV/AIDS initiatives, particularly since cross-border issues were not adequately addressed by national programs. The project’s objectives remain substantially relevant to the Country Assistance Strategies for each of the six GLIA countries, with each identifying HIV prevention and control as a key strategic priority, and most identifying regional integration as a means for controlling the spread of communicable disease.

b. Relevance of Design:
Substantial. The ICR (p. 36) presents a flow chart of the project’s results chain (based on that presented in the PAD, pp. 20-21), which logically and feasibly links the project’s activities to expected outcomes in the areas of capacity-building and service delivery, to expected coverage outcomes and the achievement of the project’s development objectives, and finally to expected health and development outcomes. Standardizing service provision across the main transport corridors for long distance truck drivers remains substantially relevant. The ICR argues that an explicit capacity-building phase for the GLIA Secretariat, the institutional home for the project, should have been identified (p. 13). A December 2009 Quality Assurance Group (QAG) review of the project criticized project design for targeting the wrong populations, citing a rapid epidemiological assessment that found fishermen, long distance truck drivers, military personnel, and female sex workers relatively neglected and at increased risk of HIV transmission; the ICR argues, however, that the project did explicitly target long-distance truck drivers and female sex workers, and that the other cited groups were clearly the responsibility of national governments (p. 14).


4. Achievement of Objectives (Efficacy) :

Establishment of HIV/AIDS prevention, care, treatment, and mitigation programs for mobile and vulnerable groups such as refugees, transport sector workers, and infected/affected populations: Modest.

Although results presented for the key performance indicators are strongly positive for three of the six member countries, data on key outcomes are presented only for those three countries. The ICR does not explain why data were not available for the DRC, Burundi, and Rwanda. The project team explained that data collection was not possible due to the conflict situations in DRC and Burundi, and that data issues related to the project were politically sensitive in Rwanda due to issues (dating to the 1994 genocide) about defining citizen populations separately from refugee populations.

Outputs:

According to the ICR (p. 19), access to a comprehensive package of prevention, care, and treatment services was virtually universal at all sites supported by the project.

HIV prevention: All nine refugee camps and surrounding communities reported uninterrupted and sufficient supplies of male condoms, meeting the target. All nine refugee camps and surrounding communities had HIV-related posters and billboards in the appropriate languages, meeting the target. All nine refugee camps and surrounding communities had functioning peer educator programs, exceeding the target of seven, working through functioning youth centers. All nine refugee camps and surrounding communities had access to functioning voluntary counseling and testing (VCT) and prevention of mother-to-child transmission (PMTCT) services, meeting the target. 5.2 million condoms were distributed to project-funded target groups, through 307 distribution outlets by 2010 (an increase from 43 available distribution outlets in 2008). 1.8 million persons were reached through information, education, and communication/behavior change communication (IEC/BCC) programs. About 14,500 trained peer educators disseminated messages to their counterparts at youth centers; those youth centers were constructed by the project. About 139,000 persons benefited from VCT programs. About 65,000 pregnant women attended antenatal clinics and received HIV test results. Almost 1,800 pregnant women benefited from PMTCT programs. An estimated 10,200 persons were placed on antiretroviral therapy (ART); all refugee sites had at least one ART center.

HIV/AIDS treatment: All nine refugee camps and surrounding communities had health workers treating sexually transmitted infections (STIs) who had been trained in syndromic management, and had appropriately trained health workers treating common opportunistic infections (OI), meeting the target in both cases.

Over 20 health centers were upgraded, rehabilitated, or constructed for the purposes of offering HIV/AIDS prevention and treatment services.

Care: All nine refugee camps and surrounding communities had trained community health workers who were actively providing home-based care, exceeding the target of seven. Home-based care was provided for over 21,000 persons.

In addition, the project funded a wide range of activities to reduce gender-based violence, mitigate sexual and economic vulnerability through income-generating activities, and improve conditions of vulnerable groups. School supplies and basic support were provided for roughly 33,600 orphans and vulnerable children (OVCs).

Outcomes:

For each of the indicators listed below, targets were not set, and data were not provided for Burundi, DRC, or Rwanda. For data that are provided on the other three member countries (Kenya, Tanzania, Uganda), the ICR points out that there were virtually no HIV services in refugee camps prior to the project, and only limited services in the surrounding communities, making it extremely plausible to attribute these outcomes to the project’s interventions (p. 18). Furthermore, interviews with beneficiaries suggest that most of the HIV-related interventions would not have taken place in the absence of the project.

The percentage of youth (aged 15-24) in the target populations reporting use of a condom during last sexual intercourse with a non-regular partner was (exact dates are not provided for “baseline” and “completion”):
• Kenya: 31.4% at baseline and 63.0% at completion for those in refugee camps, and 24.6% at baseline and 47.2% at completion for those in surrounding communities
• Uganda: 9.7% at baseline and 18.2% at completion for those in refugee camps, and 18.0% at baseline and 34.4% at completion for those in surrounding communities
• Tanzania: 39.0% at baseline and 39.0% at completion for those in refugee camps, and 28.0% at baseline and 35.4% at completion for those in surrounding communities

The percentage of respondents (aged 15-24) in the target populations who both correctly identify ways of preventing sexual transmission of HIV and who reject major misconceptions about HIV transmission or prevention was:
• Kenya: 45.1% at baseline and 35.4% at completion for those in refugee camps, and 18.3% at baseline and 34.2% at completion for those in surrounding communities
• Uganda: 32.6% at baseline and 33.9% at completion for those in refugee camps, and 38.5% at baseline and 45.7% at completion for those in surrounding communities
• Tanzania: 25.7% at baseline and 46.9% at completion for those in refugee camps, and 34.1% at baseline and 59.4% at completion for those in surrounding communities

In addition to these key project indicators, the ICR reports increases in abstinence and a “notable reduction” in high-risk sexual activity “at virtually all sites supported by the project” (pp. 18-19), citing data the percentage of never-married men and women aged 15-24 who had never had sex remaining virtually unchanged in Uganda, increasing by over 8% in Kenya (in the surrounding communities), and increasing by over 20% at all sites in Tanzania. The percentage of respondents reporting fewer sexual partners in the past year declined, with reduction ranging from 39-58% in Uganda (it is unclear what this range represents), 38-68% in Tanzania, and 42% in Kenya (at sites in the surrounding communities). The percentage of men and women (aged 15-49) who had more than one sexual partner in the last 12 months declined in both refugee camps and in surrounding communities in Kenya, Uganda, and Tanzania: from 11.9% to 3.6% in Kenyan refugee camps; from 11.6% to 6.7% in Kenyan surrounding communities; from 10.1% to 4.2% in Ugandan refugee camps; from 16.3% to 10.0% in Ugandan surrounding areas; from 32.6% to 20.1% in Tanzanian refugee camps; and from 22.4% to 8.1% in Tanzanian surrounding areas (ICR, p. 41).

The percentage of women (aged 15-49) who were forced to have sex in the last 12 months was:
• Kenya: 5.2% at baseline and 0.9% at completion for those in refugee camps, and 9.2% at baseline and 2.0% at completion for those in surrounding communities
• Uganda: 1.6% at baseline and 1.4% at completion for those in refugee camps, and 2.4% at baseline and 0.2% at completion for those in surrounding communities
• Tanzania: 3.2% at baseline and 2.4% at completion for those in refugee camps, and 1.5% at baseline and 0.2% at completion for those in surrounding communities

The percentage of men and women (aged 15-49) who had sex with a transactional partner in the last 12 months was:
• Kenya: 1.3% at baseline and 1.0% at completion for those in refugee camps, and 1.6% at baseline and 1.2% at completion for those in surrounding communities
• Uganda: 1.1% at baseline and 0.8% at completion for those in refugee camps, and 3.5% at baseline and 1.7% at completion for those in surrounding communities
• Tanzania: 14.0% at baseline and 12.0% at completion for those in refugee camps, and 4.3% at baseline and 1.6% at completion for those in surrounding communities

The ICR (pp. 19, 39) also reports a 1.5-fold increase in the number of reported sexual- and gender-based violence cases in the last three years of the project (1,196 in 2008 to 1,772 in 2010), which key stakeholders identified not necessarily as a real increase, but instead an improvement in reporting as women became more willing to come forward about cases of abuse. At the same time, the percentage of those reporting such violence who received appropriate medical attention increased by 2.5 times (from 418 in 2008 to 952 in 2010).

Accepting attitudes toward PLWHA were reported as unchanged in Uganda and improved slightly in Tanzania (no exact data are provided). In the surrounding communities there was a three-fold improvement in accepting attitudes in Uganda, but steep drops in both Kenya and Tanzania (the ICR reports 48-60%, without specifying precisely the meaning of those data).

Enhancement of prospects for coordinated approaches to HIV/AIDS prevention, care, treatment, and mitigation: Modest. Although training sessions and meetings were held, the ICR presents little evidence of outcomes resulting from those events (training effectively put into practice, new strategies or learned good practices implemented, etc.).

Facilitators were trained in the AIDS Self Assessment process: 21 in Tanzania, 21 in Kenya, and 4 in Uganda. Trainers were trained in the area of managerial capacity building: 6 in DRC, 15 in Kenya, 18 in Rwanda, 10 in Tanzania, and 6 in Uganda.

Four formal health sector interaction meetings were held between the GLIA member states to exchange information, country experiences, and effective practices, exceeding the target of two such meetings. Two formal interactions between GLIA member states were held to exchange information on HIV/AIDS prevention, care, and treatment for refugees, IDPs, returnees, and surrounding communities, meeting the target of two such meetings. Six protocols were harmonized and adopted, exceeding the target of three. According to beneficiaries and partners interviewed during the ICR mission, this harmonization of protocols has resulted (or will result) in better services for mobile populations (such as long-distance truck drivers) (ICR, p. 50).

For PLWHA networks: Seven identified networks adopted a strategy and developed an action plan for support and strengthening, and carried out an annual AIDS Self Assessment, exceeding the target of six. Of these, all met their AIDS Self Assessment targets, exceeding the target of 90%. Three formal sub-regional meetings between networks of the six countries were held to exchange information, experiences, and good practices, attaining the target of three such meetings. Two management training sessions were organized per country for member organizations, meeting the target of two such sessions.

For transport sector networks: Seven identified networks adopted a strategy and developed an action plan for support and strengthening, and carried out an annual AIDS Self Assessment, exceeding the target of six. Of these, all met their AIDS Self Assessment targets, exceeding the target of 70%. Two formal sub-regional meetings between networks of the six countries were held to exchange information, experiences, and good practices, attaining the target of three such meetings. Two management training sessions were organized per country for member organizations, meeting the target of two such sessions.

A strategy for adapted health services for truck drivers was adopted through formal consensus during a formal meeting. 21 health sites (“Wellness Centers”) with adapted services to improve access for transport sector workers were created, almost reaching the target of 22 such sites; these cites were chosen after an inventory of health services along the main road axes, and by prioritizing hot spots for disease transmission in proximity to truck stops. These Centers, located in all six countries, responded to an important gap in service provision for long-distance truck drivers and sex workers, as well as neighboring communities. All of the GLIA member countries established these Centers; the ICR notes (p. 20) that, in the absence of regional funding, it is unlikely that the fragile states (Burundi, DRC) would have attained the same results. The Centers adapted services and hours of operation to the needs of the target groups, operated out of modest containers, generated support for local councils and governments, and established referral mechanisms to neighboring health centers to improve access to STI and HIV diagnostic and treatment services.

5. Efficiency:

Efficiency is rated Modest.

No formal economic or financial efficiency analysis was conducted at appraisal. The PAD (p. 12) contains a reference to the economic analysis on HIV/AIDS that was carried out under the umbrella Multi-Country HIV/AIDS Program for the Africa Region (MAP), which included an overall assessment of the impact of HIV/AIDS on economic development poverty and showed that a reduction in AIDS-related deaths would increase the growth of GDP. That analysis, however, was more appropriate for a country-specific project than for a regional intervention.

Efficiency is therefore discussed primarily in qualitative terms. The project worked with strong, experienced institutions with demonstrated absorptive capacities, allowing the project to be implemented immediately after effectiveness. Unit cost analysis of the number of beneficiaries reached is complicated by the fact that UNHCR pooled Bank resources with other sources of funding. Approaches were standardized and good practices shared on a regional level, which produced efficiencies; for example, the Wellness Centers for long-distance truck drivers used low cost infrastructure, standardized training, and harmonized protocols to ensure the provision of standardized care. The ICR does not present specific data to support these claims of efficiency, where quantitative evidence could reasonably be expected. The bulk of project resources (over 72%) were invested directly in service provision or institutional capacity strengthening. The project contributed to individual NACs reorienting their HIV/AIDS programs toward focus on high-risk groups (ICR, p. 50), which is known to be the most efficient way of preventing the spread of HIV.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project’s objectives were Highly relevant, and its design was Substantially relevant; the project provided important complementary support to ongoing country HIV/AIDS initiatives, particularly since cross-border issues were not adequately addressed by national programs. Achievement of the first objective, establishing prevention, care, and treatment programs for mobile and vulnerable groups, was Substantial for Kenya, Uganda, and Tanzania, but because data are not provided for Burundi, DRC, or Rwanda, overall achievement of this objective must be rated Modest. Achievement of the second objective, enhancement of prospects for coordinated approaches to HIV/AIDS prevention, treatment, care, and mitigation, is also rated Modest; although networks were strengthened and approaches harmonized among all six participating countries, there is no evidence provided of outcomes resulting from these interactions. Efficiency is rated Modest, as explained in Section 5.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

Contextual risk: There is a global shift toward broader health system strengthening rather than disease-specific effort, with the bulk of global financing directed at national (rather than regional) programs. The sense of urgency surrounding HIV/AIDS globally has diminished, and there is a risk that complacency will set in. As donors grapple with the global economic downturn, there is a move toward greater selectivity in financing projects. As more and more economic and trade issues are handled through regional economic blocs, Ministries of Finance may be less likely to devote attention to single-themed initiatives.

Financial risk: The Chair of the GLIA Council of Ministers submitted a formal request and detailed concept note for a follow-up operation on March 16, 2010. The Bank declined, suggesting instead that the GLIA Secretariat submit a proposal to the Global Fund (this was done during Rounds 8 and 10, but rejected). A Partnership Consultative Group was established by the Secretariat toward the end of the project, too late to serve as an effective platform for additional resource mobilization beyond the contributions from individual member states (ICR, p. 23). The Secretariat’s inability to diversify funding sources places the project’s achievements in a “precarious financial situation” (ICR, p. 25).

Institutional risk: Investments in infrastructure and human resources will remain in the refugee camps and surrounding areas, and along the two main road corridors in East Africa. The production of two Behavioral Surveillance Surveys (BSS) provides a strong basis for planning future investments. Civil Society Organizations supported under the project will likely retain management and fiduciary capacities, enabling many to submit requests to other partners and better manage existing resources. However, the political and economic context in East Africa has evolved since project inception, with the overriding priority shifting toward strengthening the East African Community (EAC). Currently, efforts are under way to ensure that activities initiated under the project are incorporated into national HIV/AIDS strategic plans and/or transferred to other agencies and institutions. Discussions are taking place with the EAC health desk to determine the best ways to provide continuity of key activities, particularly in terms of protocol harmonization.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
The project was prepared by a strong Bank team in close collaboration with GLIA member states and several key partners, including UNHCR and the United Nations Programme on HIV/AIDS (UNAIDS). The task team facilitated the development of a common vision and a joint regional agenda among all six participating countries. It conducted an assessment of regional institutions early in the preparation process. Selection of UNHCR as an implementing partner was based on UNHCR’s close work with member states during project preparation, and on the support it provided in drafting work plans for refugee sites. The team drew key lessons from the successful Abidjan/Lagos HIV/AIDS Transport Corridor project (US$ 16.6 million, 2004-2007), which inspired the institutional arrangements for this project.

Risks and mitigation measures were well identified. For example, the risk of insufficient fiduciary capacity was well mitigated by retaining a Fiduciary Management Agency (FMA) for the first year. The risk that the GLIA Secretariat would have difficulty carrying out its full range of responsibility were mitigated through the use of specialized agencies, such as UNHCR and its subcontractors, to reach underserved groups. Uncertainty in relations between refugee communities and surrounding communities was mitigated by adding, in addition to support for refugees, IDPs, and returnee populations, parallel efforts in surrounding communities.

A Social and Gender Assessment, recognized as best practice by the Bank’s gender group, was conducted in the DRC, Tanzania, and Uganda, informing project design in a number of key areas, including the emphasis placed on empowerment of mobile populations to address the basic needs of families and minimize transactional sex, and the strategy of targeting both refugees and surrounding communities (which correctly aimed to promote equity in service provision, foster solidarity, and minimize the risk of strained relations).

However, there were shortcomings. Baseline data and targets were not available for the majority of key indicators. Several indicators were later judged to be flawed, with the MTR recommending a formal restructuring to: (i) drop HIV prevalence and social/gender outcome indicators; and (ii) refining other outcome indicators and intermediate indicators to reflect better the development outcomes. Even though the results framework was not formally restructured, the project tracked and reported on several indicators that were not included in the original results framework. Also, the Summary of the Borrower’s ICR (ICR, p. 60) argues that, for Components 2 and 3, the design did not sufficiently take into account variable national capacities and procedures. Despite a US$ 0.6 million Project Preparation Facility to facilitate recruitment of the FMA and preparation of detailed operational manuals and work plans, readiness to begin project activities was tenuous, as reflected in the one-year delay in effectiveness (due to difficulties and delays in meeting the conditions of effectiveness, setting up project accounts, and selecting intermediary agencies).

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:
There was good continuity in task management, with one field-based task team in place from inception to completion. Specialists were added, as needed, in the areas of pharmaceuticals, health, M&E, social development, and gender. Oversight of fiduciary aspects was particularly strong (ICR, p. 26), with good attention to safeguards policies and attention to gender issues. Governance and anti-corruption issues were well handled (ICR, p. 26). The task team made a concerted effort to address QAG recommendations, including providing additional support on M&E, ensuring that the second round of a Behavioral Surveillance Survey (BSS) was completed, and engaging national authorities to mainstream priority activities into national programs.

However, the task team did not succeed in restructuring the operation as recommended by the MTR. According to the ICR (p. 26), “the Bank did not provide the leadership and sustained support which was needed to rectify the weaknesses in the results framework and to recalibrate the development objectives, and outcomes.” The ICR (p. 11) notes that the failure to restructure the project was a missed opportunity to take into account early lessons and to recalibrate the results framework. The Bank also missed an opportunity to include this project in the generic restructuring of all MAP operations, to remove reference to reduction in HIV prevalence rates. The candor, realism, and consistency in Implementation Status Reports (ISRs) were mixed (ICR, p. 26). In some cases, supervision ratings tended to be more optimistic than warranted by the detailed component-specific description in the aide-memoires, and in other cases ratings may have been too negative. As noted by the QAG panel, “management attention should have been stronger” (ICR, p. 27). Management feedback on ISRs was not systematic.

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Member states demonstrated strong commitment to the project, overcoming political differences to reach consensus on a regional HIV control agenda. The GLIA Council of Ministers met regularly to guide the institution and resolve key issues. While government did not always provide funds on a timely basis to the GLIA Secretariat, in part due to their precarious financial and political conditions, by project completion there were no arrears. There were occasional delays by Governments in submitting financial reports.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
The GLIA Secretariat was given an ambitious mandate in relation to its initial capacity. Institutional strengthening efforts produced mixed results. In the area of financial management and fiduciary matters, including internal audit functions, the Secretariat performed well, ensuring a smooth transition from an FMA to performing those functions in-house. The Secretariat also put in place adequate procurement capacity, conducting procurement with few difficulties. With support from the Bank’s Global AIDS Monitoring and Evaluation Team (GAMET), the Secretariat completed an HIV epidemiological and response analysis for the six GLIA countries; based on this analysis, GLIA formulated its 2008-2012 Strategic Plan that included priority activities to be scaled up for mobile and at-risk populations.

However, there were shortcomings with the Secretariat. The ICR cites three major factors that slowed implementation, all of which stemmed from shortcomings in the Secretariat: inadequate management capacity; cumbersome flow-of-funds mechanisms, which resulted in protracted delays in channeling funds to beneficiary organizations; and weak M&E capacity, which rendered difficult the monitoring of project activities, validation of data, and assessment of impact. Even though the GLIA Secretariat was not a new structure, it was assigned substantial new roles and responsibilities; several weaknesses, including weak accountability relationships, a policy of recruiting staff based on nationality and not merit, and problems recruiting and retaining staff, hindered its performance. A Partnership Consultative Group, which could have provided a platform for effective partner coordination and additional resource mobilization, was established only toward the end of the project (ICR, p. 23).

The performance of other key implementing agencies was strong. UNHCR subcontracted organizations with a long-standing track record in working with refugees, carrying out activities on a timely basis and with good results. UNHCR worked closely with local and district health authorities, but its coordination with the NACs (which were mandated to carry out activities in surrounding communities) could have been more effective. The roles and responsibilities of GLIA country focal points were not clearly defined, and since some had other responsibilities in their respective NACs, their responsibilities toward the project were diluted (ICR, p. 33).

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The PAD (pp. 22-225) contained a detailed results monitoring framework. The GLIA M&E system was meant to function symbiotically with the six national M&E systems, to avoid duplicative efforts. The GLIA M&E Focal Points (based at the NACs) were meant to be the key interface between the GLIA Secretariat and the NACs, disseminating data to inform planning of future HIV activities, and ensuring that all routine GLIA data were of good quality. The Secretariat had oversight responsibilities for M&E planning, system training and implementation, report preparation, coordination of M&E dissemination activities, development of an operations research agenda, and use of common data with the financial management system. Some outcome indicators were vague and qualitative in nature (“increase in social and gender conditions”) or could not easily be attributable to the project (“reduction in HIV prevalence rates”). There was incomplete baseline data for many key performance indicators.

b. M&E Implementation:
The use of BSS surveys at inception and completion was best practice, generating important information about the key drivers of the epidemic, and facilitating an assessment of progress on key performance indicators; however, coverage was incomplete (only 4 of the 6 countries were covered, with Burundi and DRC excluded due to difficult country conditions, and the end-of-project survey not conducted in Rwanda due to disagreements between UNHCR and the Government on the modalities for conducting the survey). Overall, M&E capacity was weak, “which rendered difficult the monitoring of project activities, validation of the data, and assessment of impact” (ICR, p. 14). Data quality was poor, and GLIA country focal points did not devote sufficient time to validating data. Few data audits and spot checks were performed. Moreover, reporting was not done on a quarterly basis, as planned.

a. M&E Utilization:
The project team explains that there was an M&E group, composed of M&E staff of each country, that served as a forum for interchange between the M&E technicians and those making use of the data; this group met regularly throughout the life of the project. With support from GAMET, the Secretariat completed an HIV epidemiological and response analysis for the six GLIA countries; based on this analysis, GLIA formulated its 2008-2012 Strategic Plan that included priority activities to be scaled up for mobile and at-risk populations.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The project triggered OP 4.01 Environmental Assessment and OP 4.12 Involuntary Resettlement. The environmental screening category was B. Key safeguard policy issues were medical waste management at health care facilities and impacts due to construction and rehabilitation of facilities, including health care facilities. According to the ICR (p. 15), environmental issues were adequately addressed, including appropriate disposal of waste in refugee camps. The ICR does not directly state whether there was compliance with all safeguards; subsequently the project team confirmed that there was compliance with all safeguards.

b. Fiduciary Compliance:
The GLIA Secretariat maintained sound financial management systems, ensured key personnel (including an internal auditor) were in place, and produced timely interim financial and audit reports in compliance with the provisions of the DGA. The Secretariat ensured a smooth transition from the FMA to in-house capacity. The major financial issues that impeded implementation were delays by Governments in submitting financial reports and in providing counterpart contributions. All arrears were cleared by project completion. The ICR does not directly state whether audits were unqualified; the project team confirmed that all audits were clean and unqualified. The mix of procurement methods and management of procurement were “well done” (ICR, p. 16).

c. Unintended Impacts (positive or negative):
The project brought together officials from countries that had tense diplomatic and political relations. According to the ICR (p. 24), collaboration around HIV/AIDS contributed to broader engagement and improved understanding among the GLIA member states. Also, the project promoted collaboration between refugees and surrounding communities, “building trust and strengthening social capital in impoverished communities” (ICR, p. 24).

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Unsatisfactory
The ICR cites three major factors that slowed implementation, all of which stemmed from shortcomings in the GLIA Secretariat: inadequate management capacity; cumbersome flow-of-funds mechanisms, which resulted in protracted delays in channeling funds to beneficiary organizations; and weak M&E capacity, which rendered difficult the monitoring of project activities, validation of data, and assessment of impact. Even though the Secretariat was not a new structure, it was assigned substantial new roles and responsibilities; several weaknesses, including weak accountability relationships, a policy of recruiting staff based on nationality and not merit, and problems recruiting and retaining staff, hindered its performance. A Partnership Consultative Group, which could have provided a platform for effective partner coordination and additional resource mobilization, was established only toward the end of the project (ICR, p. 23). Also, the roles and responsibilities of GLIA country focal points were not clearly defined, and since some had other responsibilities in their respective NACs, their responsibilities toward the project were diluted (ICR, p. 33). 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Regional investments can add value to national investments and maximize impact, particularly for populations that tend to be neglected by national governments. Individual countries may not have the incentives and resources to address the needs of refugees located on their territories, nor of long-distance truck drivers who move between their territories.


Addressing the needs of mobile and vulnerable groups and surrounding communities in an integrated manner avoids creation of distortion and perceptions of unfairness. Refugee camps can serve as poles of development for the provision of health and related social services.

Frequent data collection, analysis, and dissemination of results are critical when dealing with mobile populations. In refugee camps, where there may be constant inflows of new arrivals, needs may change rapidly. Surveys that generate “just-in-time information” are particularly useful to implementers and policymakers in these situations.

Regional projects are inherently complex, presenting an imperative for simplicity of design. Sequencing of activities, simple flow-of-funds mechanisms, focus on a relatively few number of key activities, reliance on established institutions, and the use of an explicit initial capacity-building phase can enhance the probability of success.

Management should provide strong support and guidance on project restructuring, and should create an environment that rewards task teams for successful restructuring of operations.

Given that regional projects often involve multiple institutions and countries, planning for sustainability of regional initiatives should begin early in the project cycle. Assisting beneficiary networks or other local groups to take over management of facilities support by a regional project should be given priority and agreed upon by the MTR.


14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is clear, concise, and evidence-based. It (p. 36) presents an effective graphic presentation of the project’s results chain, illustrating the links between activities funded and achieved results. Its discussion of attribution of observed results to the project’s interventions (p. 18) is straightforward and effective. It provides (p. 32) appropriate cautionary notes regarding the data produced by the project, particularly the BSS. However, the ICR does not provide clear statements on compliance with safeguards and fiduciary performance.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions