Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Health And Nutrition Support Project


  
1. Project Data:   
ICR Review Date Posted:
06/08/2012   
Country:
Mauritania
PROJ ID:
P094278
Appraisal
Actual
Project Name:
Health And Nutrition Support Project
Project Costs(US $M)
 11.0  11.2
L/C Number:
C4181
Loan/Credit (US $M)
 10.0  10.8
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  06/01/2006
 
 
Closing Date
12/31/2009 06/30/2011
Sector(s):
Health (63%), Central government administration (25%), Non-compulsory health finance (10%), Sub-national government administration (2%)
Theme(s):
Nutrition and food security (29% - P) Health system performance (29% - P) Population and reproductive health (14% - S) Other communicable diseases (14% - S) Child health (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judith Hahn Gaubatz
Judyth L. Twigg Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
According to the Project Appraisal Document (PAD, page 10), the overall objective of the project was:

  • to strengthen the health system and its capacity to improve the health and nutrition status of the population, notably of women, children and the poor, as it will support the implementation of the Government Program for the health and nutrition sectors during the period 2006-2008.

The PAD (page 10) also articulated the specific objectives as follows:
  • to improve access to basic health services in underserved areas;
  • to improve the equitable allocation of resources to underserved areas;
  • to strengthen health sector management to raise efficiency; and
  • to enhance and expand community-based communications for improved nutrition.

The Financing Agreement (page 5) includes the same articulation of the overall objective, but not the specific objectives. Therefore, as the PAD statement of objectives (both overall and specific) provides more clarity on the intended outcomes for assessment, it is used as the basis for this review.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
Note: Components 1-4 were to be implemented by the Ministry of Health and Social Affairs (MOHSA). Component 5 was to be implemented by the State Secretariat for the Promotion of Women (SECF).

1: Develop human resources and improve their geographical distribution (Appraisal: US$2.0 million; Actual: US$0.46 million): This component aimed to strengthen sector capacity to manage human resources, including through improved intersectoral coordination. Activities included: capacity-building activities for the Directorate of Human Resources, support for a human resources management system, establishment of a network system for information sharing, piloting of an accountability system for health care providers, coordination with training institutes, revision of curriculum and training programs, and integrated supervision (central, regional, and decentralized levels) of human resources.

2: Ensure adequate sector financing and equitable allocation of resources for the poor and for underserved geographical areas (Appraisal: US$1.5 million; Actual: US$0.28 million): This component aimed to improve existing processes and methods for financing, and to strengthen measures to ensure financial accessibility and utilization of services by the poorest and most vulnerable. Activities included: technical assistance for annual expenditure reviews and national health accounts, subsidization of essential health services for the poor, and support for establishing alternative financing systems at the community level.

3. Improve health sector management to raise efficiency (Appraisal: US$1.0 million; Actual: US$1.68 million). This component aimed to support the sector-wide approach, and to develop management capacity of health sector staff. Activities included: support for establishment of clear objectives, roles and relationships among the partners; support for improved coordination and execution of planning, budgeting, procurement, and financial management; and development of monitoring and evaluation modalities.

4. Improve the accessibility to quality and affordable health services in underserved areas (Appraisal: US$3.5 million; Actual: US$4.01 million). This component aimed to improve access to and quality of basic health services, and to raise demand for services. Activities included: construction of health posts in areas of low accessibility, support to the community approach for reaching low-access populations, improvement of the availability of drugs, and awareness-raising emphasizing prevention and behavioral change. These activities would focus on the reduction of child mortality (through the Integrated Management of Childhood Illnesses approach), the reduction of maternal mortality (through the improvement of emergency obstetrical and neonatal care), and the reduction in incidence of schistosomiasis.

5. Enhance and expand community-based communications for improved nutrition (Appraisal: US$2.0 million; Actual: US$2.73 million). This component aimed to improve access to basic essential health and nutrition services (i.e. micronutrient supplementation, de-worming, and distribution of bed-nets) and strengthen the capacity of SECF to plan, monitor and evaluate nutrition programs. Activities included implementation of community-based nutrition communication strategies, implementation of salt iodization activities, and training of community agents.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project cost:

  • The actual project cost was US$11.2 million, according to the project team. Disbursements for Components 1 and 2 were lower than planned, primarily due to capacity constraints. The project team noted that there were challenges in coordination among the Directorates, which affected project implementation.

Financing:
  • The actual loan amount was US$10.8 million.
  • The project was part of a sector-wide approach program (SWAp), although it focused on specific aspects of the national program rather than pooling funds with other donors. The ICR does not provide specific details on other donor funding, although it reports (page 35) that the share of IDA funding in the sector (total spending of government and donors) was 17%, compared to all other donors' share of 9% (no specific year provided).

Borrower contribution:
  • Counterpart funding of US$0.84 million was provided with no major issues reported.

Dates:
  • The closing date was extended from December 31, 2009 to June 30, 2011. Due to the August 2008 coup, Bank operations in the country were suspended for a period of one year. When operations were resumed in September 2009, only three months remained in the original project period. Thus, the Bank and Government agreed to extend the closing date to enable completion of activities.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial. The objectives were fully consistent with the health and nutrition needs of the population, particularly for women, children and the poor, as reflected by a child mortality rate of 116 per 1,000 live births; infant mortality rate of 74 per 1,000 live births; maternal mortality rate of 747 per 100,000 live births; and chronic malnutrition affecting nearly one-third of children.) The objectives also appear relevant to the Government's priorities per the National Health and Social Action Policy (2005-2015) and the National Nutrition Development Policy (2005-2010), both of which had a strong poverty focus and aimed to accelerate outcomes in health and nutrition, although policies specifically related to the project are not clear. The objectives are also consistent with the Bank's Country Assistance Strategy (FY08-11), current at project closure, which emphasizes continued focus on improving access to health and nutrition services, while increasing focus on financial sustainability issues. Lastly, the objectives were highly relevant to the Bank's corporate priorities of achieving the Millennium Development Goals.

b. Relevance of Design:
Substantial. The project design was built upon ongoing partnerships and the existing sector-wide approach (SWAp), while focusing on specific areas of responsibility for the Bank. These included public expenditure analysis, capacity-building, as well as nutrition policy. The project activities appropriately addressed supply (i.e. construction of health centers) and demand (i.e. community outreach, affordability) issues, as well as measures to increase sector management capacity. A significant proportion of project funds were to go towards treating diseases common among the poorest population groups (including schistosomiasis). While these activities were likely to achieve the intended intermediate outcomes (i.e. increased access to services, reduced inequities, improved efficiency) and contribute to overall outcomes (i.e. improved health and nutrition outcomes), the planned time frame of the project (three years) was likely too short.


4. Achievement of Objectives (Efficacy) :

Specific objectives:
Note: For the first three objectives below, the activities were part of a SWAp. The ICR does not provide specific details on other donors' activities (or financing amounts), although, as noted previously, it reports (page 35) that the share of IDA funding in the sector (total spending of government and donors) was 17%, compared to all other donors' share of 9% (no specific year provided).

To improve access to basic health in underserved areas.
Modest, due to mixed evidence of outcomes. Note: The health facilities financed by this component were selected based on low accessibility.

Outputs
  • Construction and equipping (medical equipment, maintenance tools, solar power) of 13 health posts and 1 health center.
  • Conducting of three mass de-worming campaigns for the treatment of schistosomiasis, including development of awareness communication and provision of mebendazole drugs.
  • Training abroad for 4 chief doctors of health centers, and training locally for 170 health personnel (on issues such as management of diarrhea, neonatal care ).
  • Provision of 8 ambulances to health posts.
  • Provision of 40 motorcycles to the head nurses of health posts.

Outcomes
(For all figures below, the baseline year was 2006 and the final year was 2011).
  • The rate of mass treatment for schistosomiasis reached 100% in all four regions, from a baseline of 0%.. This represented the treatment of 209,340 children in primary schools, 28,610 children in Koranic schools, and 16,000 at-risk adults. Although data on prevalence would have been a more robust indicator of improvements in health, it is likely that the high rate of treatment would lead to decreased prevalence rates.
  • The percentage of pre-natal centers offering subsidized services for pregnant women increased in all four regions, with targets fully or nearly achieved in three of the four regions. (Hodh El Chargui: 44% to 59% (target: 60%); Hodh El Gharbi: 48% to 57% (target: 66%); Assaba: 63% to 88% (target: 69%); Guidimaka: 73% to 77% (target: 79%).

However,
  • Evidence of improved coverage of pregnant women receiving two doses of a malaria prophylaxis (sulfadoxine-pyrimethamine - SP) is inadequate, due to unclear baseline figures. The ICR (page iii) reports baseline coverage of 0%, therefore indicating improvements in all seven regions. However, the PAD (page 38) reports baseline figures as follows: Hodh El Chargui: 51%; Hodh El Gharbi: 81%; Assaba: 50%; Traza: 61.5%; Guidimaka: 76%; Brakna: 86%; Tagant: 51%. End-project levels of coverage were as follows: Hodh El Chargui: 45% (target: 58%); Hodh El Gharbi: 32% (target: 88%); Assaba: 52% (target: 57%); Traza: 54% (target: 66%); Guidimaka: 50% (target: 83%); Brakna: 55% (target: 90%); Tagant: 29% (target: 57%).

To improve the equitable allocation of resources to underserved areas.
Negligible, due to lack of evidence on resource allocation outcomes.

Outputs
  • Training of staff in monitoring public spending on health, implementation and monitoring of national health accounts, financing of community health care, and management of community development programs. In addition, 53% of health committee members were trained on the cost recovery system, achieving the target of 50%.
  • Completion of annual expenditure reviews of the health sector and national health accounts.
  • Establishment of 20 health mutuelles (risk-sharing financing mechanisms at the community level), surpassing the target of 15.
  • Implementation of awareness campaigns for communities to renew health committee members on the basis of newly established criteria.

Outcomes
  • Health sector capacity in resource allocation was likely improved as a result of the outputs; however, the extent to which the improved capacity led to more equitable allocation of resources is unclear.

To strengthen health sector management to raise efficiency.
Modest, due to lack of information on efficiency outcomes.

Outputs
  • Development of the National Health Development Plan, including mobilization of partners.
  • Conducting of annual reviews of the health sector to monitor results, and planning workshops to develop action plans.
  • Signing of a Memorandum of Understanding among donors (for improved coordination) and creation of a Partner Group for the Development of Health, Social Action and Nutrition sectors.
  • Development of tools for integrated supervision (i.e. with Human Resources Department, DSSB) of the health sector.
  • Collection of health information for the years 2008, 2009, and 2010.
  • Training of staff on human resource management and use of the national health information system.
  • Updating of all health personnel files, in order to further develop human resources and improve geographical distribution.
  • Conducting a study to assess premiums and the introduction of performance bonuses.
  • Updating of human resource development strategic plan.
  • Holding 9 coordination meetings with regional health departments and other Ministries (i.e. Public Service, Finance, Education).

Outcomes
  • Health sector capacity in management was likely improved as a result of the outputs; however, the extent to which the improved capacity led to increased efficiency is unclear.

To enhance and expand community-based communications for improved nutrition.
Substantial, due to evidence of improved nutrition practices. Note: Activities under this component were not part of the SWAp, and therefore outcomes are attributable to the Bank's specific support.

Outputs
  • Development of communication strategy for behavior change in nutrition.
  • Establishment of the National Council for the Development of Nutrition, chaired by the Prime Minister.
  • Establishment of 36 community nutrition centers in four regions. These centers provided services such as micronutrient supplementation, monthly growth monitoring of children, and cooking demonstrations.
  • Provision of micronutrients (iron folate and Vitamin A supplements) and drugs (mebendazole) to support national programs.
  • Testing of salt iodization and conducting of awareness campaigns on the consumption of iodized salt.
  • Training of staff on community nutrition, awareness raising, and community mobilization.

Outcomes
(For figures below, the baseline year was 2006 and the final year was 2011, unless otherwise noted).
  • The percentage of women who practice exclusive breast feeding for the first six months after birth increased in project regions, although falling short of targets in all but one region. (Nouakchott: 15.2% to 40.2% (target: 44.1%); Nouadhibou: 21.9% to 46.9% (target: 25.8%); Gorgol: 17.6% to 42.6% (target: 42.7%); Assaba: 20.3% to 45.3% (target: 79.5%); Hodh El Garbi: 32.9% to 57.9% (target: 80%)).
  • Coverage of Vitamin A supplementation for children under five years old increased from 43.15% to 103.42%, surpassing the target of 78%.
  • The consumption of iodized salt in project regions increased from 28.3% in 2008 to 49.9% in 2010. No target was provided, and the source of the data is not clear.


Overall objective:

To strengthen the health system and its capacity to improve the health and nutrition status of the population, notably of women, children and the poor.
Modest, based on the modest or negligible achievement of two out of the four specific objectives above, and limited information on other outcomes relevant to the overall objective.

5. Efficiency:

Modest. The ICR does not provide sufficient evidence on the efficient use of project resources i.e. that the project was implemented at least cost or that the interventions as implemented were cost-effective. According to the ICR, there was a suspension of activities for one year during the project period, implementation and procurement delays due to capacity constraints in MOHSA, and delayed attempts by the Bank to restructure the project.


A project-specific economic and financial analysis was not conducted for the PAD, for reasons which included the following (PAD, page 23): i) the overarching goal of the project was to assist the Government to achieve the MDGs by means of "well proven cost effective strategies"; and ii) a public expenditure review had been recently conducted and "provided relevant information on and analysis of the sector policy from an economic perspective."

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Moderately Unsatisfactory, due to Substantial relevance of the objectives and design, and Substantial achievement of the specific objective to enhance community-based nutrition communication. However, there was Modest or Negligible achievement of the specific objectives to improve access to basic health in underserved areas, improve the equitable allocation of resources to underserved areas, and strengthen health sector management to raise efficiency due to limited information on outcomes (i.e. the extent to which the improved capacity led to more equitable allocation of resources or increased efficiency). The overall objective to strengthen the health system and its capacity to improve the health and nutrition status of the population was also rated Modest. Lastly, there was Modest efficiency in the use of project resources.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

Political instability and security problems in the region (due to the presence of Al-Qaeda) continue to post significant risks for project sustainability. Sector capacity remains inadequate and the Government's commitment to continuing training and capacity building activities is uncertain. The ICR reports that the platform for service delivery in rural areas has been discontinued due to lack of government funding.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
The project drew upon experience from the previous Health Sector Investment Project (US$17 million, 1998-2004), which adopted a sector-wide approach and emphasized strong donor collaboration, and the Nutricom Project (US$3.6 million, 1999-2005), which piloted the community-based approach and brought out capacity needs of the country. It was also based on extensive analyses of health sector issues, government strategies, donor activities, and the Public Expenditure Review in 2004. Extensive discussions were carried out with donors to coordinate interventions and aim for common objectives as laid out in the Government's policies.

However, the project design was very complex, with numerous activities to be implemented by agencies with inadequate capacity, in a relatively short time frame. Although capacity constraints were clearly known from the prior experience with the health sector project, they were still not effectively addressed; indeed the risk of weak institutional capacity was flagged in the PAD as a Moderate risk, rather than Significant or High. Political instability was also assessed as only a Moderate risk, with mitigation measures not adequately identified. The availability of institutions, and individuals, to carry out specialized activities such as community mobilization and nutrition interventions, was not properly determined. The M&E framework was weak, as indicators were inadequate for measuring the relevant outcomes (see Section 10 below).

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
During the Mid Term Review (MTR) in March 2008, the project team proposed an in-depth restructuring of the project in order to address the following problems: i) weak M&E framework, namely inadequate indicators; ii) low disbursement levels for components 1 and 2; iii) inadequate coordination between the components; iv) lack of integrated planning; and v) weak health information system. The proposed restructuring reformulated objectives (with clearer indicators) and streamlined the project components. However, due to the coup in August 2008, the restructuring process was put on hold and project implementation was suspended. After Bank operations were resumed in September 2009, the Bank proposed that the Government submit a formal request for restructuring; however, the submission of the request was delayed due to non-compliance with a financial audit. By the time the Bank received the request, the project period was nearly ended and the Bank decided not to proceed with restructuring. While the Bank team's restructuring efforts reflected the intensive and pro-active approach taken during supervision, nevertheless, the major inadequacies of the project were not resolved.

In addition, there was high turnover in task team leadership (four), which the ICR reports (page 8) had "an adverse impact on the implementation of the project, such as creating a certain lack of continuity and ownership, and different approaches to supervision." No major fiduciary problems were reported, with the exception of some weaknesses discussed in Section 11 below .

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
At the time of project preparation, the Government demonstrated strong commitment to the project, as reflected by its active participation in background analyses and appointment of a donor coordinator. However, political instability prevailed throughout the project period. There were two coup d'etats (August 2005 and August 2008), the latter of which led to the suspension of operations for one year. There was high turnover in political leadership as well, with four Prime Ministers, eight Ministers of Health, and four Directors of Financial Affairs.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
While the MOHSA had prior experience in implementing Bank projects, most Directors, including the Director of Financial Affairs within MOHSA, were new and inexperienced with Bank procedures. This led to a slow start in implementation. The ICR reports (page 20-21) that prior to the mid-term review, low disbursement in Components 1 and 2 reflected the "lack of ownership and leadership from the MOHSA Directorates in charge of their implementation." The project team also reports that significant capacity constraints, particularly with regard to coordination with other Directorates, hampered implementation of activities. Some fiduciary difficulties were experienced, as discussed in Section 11 below.

While the SECF was overall effective in implementing the project's nutrition activities, the ICR notes (page 8) that there were difficulties in recruiting nutrition staff with proper education and experience. In 2007, the SECF was converted into the Ministry of Social Affairs, Children, and Families. The ICR (page 7) notes that this was a "positive development as it provided a better institutional anchorage for the implementation of the nutrition component of the project." According to the project team, the elevation of the SECF from an MOH unit to a full Ministry increased its leverage and visibility and thereby it was more effective in drawing attention to nutrition issues.

Lastly, there were very few institutions existing to implement the community mobilization and development activities.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The PAD appropriately notes that health outcome indicators would not be appropriate given the short time frame of the project and the low likelihood of carrying out timely demographic and health surveys to capture the relevant data. However, the selected indicators, which were mostly related to process and outputs instead, were still inadequate for measuring outcomes. As noted in the ICR (page 16), indicators for increased access were too narrow, and indicators for equitable allocation of resources and strengthened management did not allow assessment of outcomes. The PAD (page 20) identifies various M&E elements to be included in the M&E framework; however, a monitoring and data collection plan is not provided in detail. Baseline data were not available for all indicators.

b. M&E Implementation:
While the M&E design included activities to strengthen sectoral monitoring and information systems, it is not clear whether these activities were effectively implemented. A beneficiary assessment mentioned in the PAD (page 20) does not appear to have been conducted. With regard to the monitoring of the project itself, the ICR reports (page 8) that until the coup in August 2008, the Department of Financial Affairs had been collecting data "more or less regularly" according to the project indicators. After the resumption of operations, evaluation specialists were recruited and helped to ensure that data was collected regularly.

a. M&E Utilization:
The ICR (page 9) reports that data collected were analyzed on a regular basis, for example with outputs being discussed with officials in the rural areas on a quarterly basis. However, "the data collected was not utilized for any decision making by the government."

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The Category B project triggered OP 4.01 Environmental Assessment and OP 4.12 Involuntary Resettlement due to potential negative environmental and social impacts related to the construction of health centers (i.e. resettlement due to land acquisition), as well as medical waste management issues. The ICR reports (page 9) that there were "no significant deviations or waivers from the Bank's safeguards and fiduciary policies and procedures during the implementation of the project."

b. Fiduciary Compliance:
Financial management: There were no major problems reported on financial management. The ICR (page7) refers to a project restructuring request that was delayed due to non-compliance with the financial audit; however, no further details are provided. The project team confirmed that the matter was likely resolved as the project extension request was approved. According to the ICR, there were weaknesses in the internal control system, especially in the accuracy of the supporting documentation and the long delay in producing justification of activities by the regions.

Procurement: Procurement in MOHSA experienced difficulties, such as lengthy processes, re-advertisement of several contracts, and turnover in staff. Procurement in SECF was adequately carried out, as evidenced by the satisfactory execution of the civil works component (including the readiness of bidding documents of medical centers prior to project effectiveness).

c. Unintended Impacts (positive or negative):
Positive: The project contributed to renewing the engagement of the government for nutrition policy development, as reflected by the posting of a technical advisor in the prime minister's office for nutrition.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Borrower Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
  • The ICR (page 22) appropriately notes that political stability in the country is important for smooth implementation and success of a project. The political instability that prevailed during this project led to frequent changes in key staff. Therefore, IEG would advise that the risks due to political instability should be carefully assessed, with adequate mitigation measures identified.
  • Related to the lesson identified in the ICR (page 22), the M&E design should include indicators that are closely linked to project activities and adequately measure results relevant to the project objectives. This project's key indicators were primarily related to process and outputs, and thus hindered the ability to clearly assess outcomes.

14. Assessment Recommended?

Yes
Why?
To verify outcomes and learn lessons particularly with regard to the nutrition activities.

15. Comments on Quality of ICR:

The ICR was satisfactory, although a shortcoming was the lack of sufficient information on environmental management and safeguard compliance.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions