Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Social Sector Support Project


  
1. Project Data:   
ICR Review Date Posted:
03/31/2014   
Country:
Sao Tome & Principe
PROJ ID:
P075979
Appraisal
Actual
Project Name:
Social Sector Support Project
Project Costs(US $M)
 7.10  9.46
L/C Number:
C3902, CH088
Loan/Credit (US $M)
 6.50  9.86
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  05/18/2004
 
 
Closing Date
06/30/2009 12/31/2012
Sector(s):
General education sector (47%), Health (26%), Other social services (14%), Central government administration (13%)
Theme(s):
Education for all (33% - P) HIV/AIDS (17% - S) Population and reproductive health (17% - S) Participation and civic engagement (17% - S) Administrative and civil service reform (16% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Erik A. Bloom
Judyth L. Twigg Lourdes N. Pagaran IEGPS2

2. Project Objectives and Components:

a. Objectives:
The financing agreement (page 24) states that:


    "The objective of the Project is to contribute to improving the delivery of basic health and education services with a focus on greater and more equitable access, better quality, and improved local governance." This objective is repeated in the PAD.

    The evaluation will focus on the project's contribution in improving basic health services and basic education services, defined by improvements in equitable access, quality, and local governance.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

The project consisted of four components:

Component 1: Delivery of Basic Education Services (appraisal: US$ 2.8 million, actual: US$ 3.1 million). Support to the Ministry of Education and Culture to improve the management and integration of the primary education system. The component was to include school construction, teacher training and the provision of pedagogical materials, and the provision of grants to non-governmental organizations (NGOs) to finance community-driven initiatives.

Component 2: Delivery of Basic Health Services (appraisal US$ 2.3 million, actual US$ 1.6 million plus US$ 2.1 million in additional financing). Support to the Ministry of Health to strengthen the delivery of basic health services at the district level. Activities were to include construction, rehabilitation, and the provision of equipment in two districts to provide clinical maternity services. The component also was to support training for health sector providers, strengthening of management capacity, information campaigns, and grants to NGOs to finance community-driven initiatives.

Component 3: Cross-Sectoral Social Issues: HIV/AIDS and Malaria (appraisal US$ 0.5 million, actual US$ 0.5 million). Support to the public sector in implementing the national program to combat HIV/AIDS and Malaria. The project was to support a variety of HIV/AIDS prevention and control activities (provision of condoms, HIV/AIDS testing, behavior change communication, etc.) and malaria control activities. Most activities were to be carried out by NGOs.

Component 4: Program Management and Implementation Support (appraisal US $0.7 million, actual US$ 1.5 million). This component was to support the activities of the implementing agency, including monitoring and evaluation.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost and Financing: The project was appraised at US$ 7.1 million, with US $5.0 million from an International Development Association (IDA) credit, US$ 1.5 million from an IDA grant, and $ 0.6 million from the government. Actual financing was US$ 5.11 million from the IDA credit, US$ 1.65 million from the IDA grant, and US$ 0.6 million from the government. There is no explanation in the ICR as to why IDA financing exceeded the expected amount. The project included US$ 0.4 million in physical contingencies. While these were fully disbursed, the ICR does not explain where they were used.

In May of 2010, the project received an IDA grant of US$2.1 million as additional financing, designated solely for the health sector (Components 2 and 3) to cover scaled-up of rehabilitation of health facilities, institutional support for the Ministry of Health (including completion of a Health Information System and new policies on health financing and human resources), residences for health care staff in remote areas, training and capacity development in the Ministry of Health, and support for community-driven initiatives to build latrines. This amount was fully disbursed. At the additional financing, some targets were adjusted and some new indicators were added. According to the ICR (page 4), the main objective of the additional financing was to consolidate achieved gains and ensure sustainability in the face of the global financing crisis.

Borrower Contribution: The Borrower made in full the planned US$ 0.6 million contribution.

Dates: The project was approved by the Board of Executive Directors on May 18, 2004 and became effective on December 15, 2004. Its original closing date was June 30, 2009, and the actual closing date was December 31, 2012.. The project was first extended to June 30, 2010 (the date of the extension is not recorded in the ICR) to allow for conclusion of construction activities. On May 4, 2010 the project received additional financing and was extended until December 31, 2011. The project was later further extended to December 31, 2012 (the date of this extension is not recorded in the ICR) to conclude civil works activities.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Relevance of Objectives is rated High.

At the time of appraisal, although public spending on health and education were relatively high (about 27 percent of total Government expenditure), there were significant inefficiencies. Education services were seen as having poor quality, with insufficient learning time. Almost one-third of public schools were operating on a triple shift system giving students only 2-3 hours of class time daily. Attrition was high, with only 78% of students who started in first grade reaching fourth grade, and only 47% reaching sixth grade. Pedagogical techniques and curricula were judged to be out of date. In the health sector, malaria was the leading cause of mortality, especially among children under age five. The number of HIV/AIDS cases was rising, with prevalence among pregnant women increasing from 1% in 1993 to 5.6% in 1996. The project's objectives were highly relevant to country conditions at the time of appraisal.

The Bank's strategy (Interim Strategy Note for the Democratic Republic of Sao Tome and Principe, approved in April 2011) reinforces the Bank's support for improving the country's competitiveness and assisting the country in managing revenues from natural resources. At the same time, it emphasizes the importance of improving the quality of basic education as an important constraint, given the current high level of enrollment. For health, the Strategy Note emphasizes the importance of ensuring equitable coverage, improving the quality of health services, and promoting greater decentralization. The project's objectives remain highly relevant to current Bank strategy.

The government's strategy, outlined in the Second National Poverty Reduction Strategy (2012-2016), includes building human capital and strengthening social services as its Pillar III. This includes efforts to improve the role of local government and communities in education and to enhance coverage and quality in both education and health. The project's objectives remain highly relevant to current Government strategy.

b. Relevance of Design:

Relevance of Design is rated Modest.

The objectives focused on improving the delivery of basic health and education services, focusing on equity and access, quality, and local governance.

In education, the project supported a number of activities that are generally associated with improved quality, such as extending the school day, teacher training, and improving the quality and relevance of education materials. The project also included activities that would plausibly improve access both by building new schools in remote areas and introducing new policies, such as improved stipends for teachers in remote areas. The initiatives to improve access would also support increased education quality (and vice versa). Education governance was supported through expansion of a community-led initiative program as well as a sub-component to strengthen the Ministry of Education.

In health, the project included activities to strengthen the capacity of the Ministry of Health, including support to the Ministry's strategic management, human resources, and information systems. The steps taken to improve quality included improving training for health workers. To increase access, the project focused on diseases that affected the poor (particularly malaria) and on providing services to the community.

However, it appears that the project's planned activities provided little consideration to the objective of improved local governance in either the education or health sectors. This shortcoming weakened the relevance of design.

The results framework largely captured these interventions with intermediate outcomes in both sectors, with the exception of improved local governance . Given the projected length of the project (five years) and the lack of many types of final outcome data (for example, student achievement), it was neither possible nor desirable to include longer-run final outcomes in the results framework.


4. Achievement of Objectives (Efficacy) :


The PDO lists three objectives (more equitable access, better quality, and improved local governance) for two different types of services (basic education and basic health). This evaluation will review all six aspects of the PDOs.

Basic Education Services

More equitable access to basic education services is rated High, due to the project's contribution to impressive gains in primary school enrollment.

Outputs

The project provided support to construct 22 new classrooms, located in poorer, rural areas (ICR, page 15) as well as support to local NGOs for community-led development initiatives including building new classrooms (ICR, page 21). In addition, the project supported the rehabilitation of schools, both through the government and through grants to NGOs. The number of additional students in primary education from underserved areas increased by 1,541 compared to the original target of 350 (intermediate outcome, provided by the project team). The introduction of automatic promotion of students from grades 1 to 2 and the de facto automatic promotion from grades 1 to 4 (page 24 of ICR) is in line with international best practice and makes a direct contribution to reduction of repetition.

Outcomes

Sao Tome and Principe saw an impressive growth in the coverage of basic education, with the survival rate in primary school rising from 47 percent in 2004 to 67 percent in 2009 (compared to a target of 65 percent) and to 116 percent in 2011 (the rate over 100 percent indicates that students are repeating instead of dropping out of school).

This increase is also evidenced by an increase in the grades 4 to 5 transition rate from 60 percent in 2004 to 84 percent in 2009. The World Bank's Country Strategy reported that by 2011, universal basic education had essentially been achieved. The ICR (page 23) also argues that Sao Tome and Principe has the highest completion rate among a group of similar sub-Saharan African countries. The basic education repetition rate decreased from 23 percent in the 2004-05 school year to 12 percent in 2010-11. This is particularly impressive given that basic education added an additional grade, and this sort of decline is rarely seen in other education systems. The ICR provides a convincing argument that the project's activities contributed to this outcome (page 24).

There are likely to be many factors behind this increase in education coverage, including economic growth and the growing role of natural resources in the economy. However, it is unlikely that these, by themselves, would have led to such a significant increase in coverage. While other development partners have supported the country, this project's support appears to have been critically important in supporting the sector.

The activities and outputs related to education quality and education governance are also likely to have contributed to access, by making schools more attractive to the poor. Two elements supported by project are likely to have played a major role. The first, the inclusion of grade 6 in the primary cycle, is likely to have contributed to the increase in coverage for that grade and thus for the entire grade 1-to-6 period. The second is the introduction of automatic promotion in the early grades.

Better quality of basic education services is rated Substantial, due to the project's support to decreased repetition and increased student contact time.

Outputs

The project supported a number of changes to the education sector that are generally associated with efforts to increase quality. These include the successful increase of the primary education cycle from five grades to six grades and the reduction of triple-shift schooling, from 27 percent of schools to 8 percent, compared to a target of 0, which led to an increase in the effective learning time per day from 2-3 hours per day to 4 hours per day (intermediate outcomes, provided by the project team). In addition to those reforms, the project supported the training of 869 teachers, which was fewer than the 1,000 teachers originally planned. Teacher training can increase education quality, but the ICR does not explain the nature of this training, nor does it provide evidence on how this training was implemented in the classroom. Finally, working with the Gulbenkian Foundation (a Portuguese NGO), all students received textbooks, compared to 50 percent of students receiving text books before the project (intermediate outcomes, provided by the project team). The provision of textbooks has generally been associated with increases in education quality. Another important element was the introduction of automatic promotion of students from grades 1 to 2; this is in line with general international practice and likely to contribute both to improved quality and equitable access.

Outcomes

The PAD (page 5) identifies the reduction of repetition as the primary goal in increasing quality; however, this is generally seen as an indicator of access and coverage.

The ICR confirms that the country does not have a learning assessment system; such a system is difficult and expensive to establish in a small, poor island nation. Instead, the outputs outlined above are taken as important indicators of improvements in quality, particularly as the objective was to improve the quality of education services (rather than improvements in learning outcomes). This includes the elimination of triple shifting and the increase in student contact time. The improvement in textbook distribution is also an increase in the quality of services. Finally, the teacher training may be seen as a potential increase in teacher quality.

Improved local governance for basic education services is rated Negligible, due to the lack of evidence on both outputs and outcomes.

Outputs

The project financed support to local NGOs to implement activities in the education sector. These NGOs implemented initiatives designed by the community. In addition, the project provided support for the Ministry of Education and other areas of central infrastructure, although this is not likely to have had much impact on local governance. The ICR does not report any other activities related to improved local governance.

Outcomes

The ICR does not report any indicators or evidence of improved local governance in the education sector or how the community-developed initiatives contributed to local governance. According to the results framework, only 30 percent of the planned financing for community-driven initiatives was disbursed, although the ICR does not explain why this was the case. 95 percent of planned sub-projects were successfully completed, although the ICR does not define "successful completion" or clarify the distribution of sub-projects between the education and health sectors.

Basic Health Services

More equitable access to basic health services is rated Substantial.

Outputs

The project supported the construction and rehabilitation of 12 health posts and centers, all of which were located in rural areas. This activity was complemented by support provided by NGOs that were financed by the project (ICR, page 15). In total, 20 health facilities were constructed or rehabilitated (intermediate outcomes, provided by the project team). The project also supported the construction of residences for health sector staff in remote areas, although the ICR does not indicate how many were built or how many staff were housed in the residences (ICR, page 22).

The project also supported NGOs for community-based interventions against HIV/AIDS and malaria, including HIV/AIDS testing, distribution of insecticide-treated bed nets, and information campaigns, among others (ICR, page 21).

Outcomes

The number of people with access to a basic package of health and nutrition services increased from 78,000 in 2009 to 105,000 in 2012 (information provided by the project team).

Better quality of basic health services is rated Substantial.

Outputs

The project supported improvements in the Health Training School. The project also supported the training of 165 health personnel, which exceeded the original target of 150 (intermediate outcomes, provided by the project team).

Outcomes

Sao Tome and Principe has seen impressive gains in health status. Intermediate outcomes include an increase in the measles immunization rate from 84 percent (2004) to 100 percent (2012), compared to a target of 95 percent, as well as as an increase in children receiving deworming treatments in school, from 50 percent (2004) to 99 percent (2009). The ICR is not clear on the extent to which the project played a role in distributing relevant inputs to the health sector, although it is clear that project's support for training and its expansion of the health network did play a role in reaching these targets.

In addition, the country saw an increase in the number of women receiving at least four antenatal visits, to 92 percent compared to a target of 95 percent for 2012 (the baseline is expressed in different terms, although it is clear that coverage did increase). It also saw an increase in the percentage of women who had attended births, from 70 percent (2004) to 93 percent (2012), compared to a target of 90 percent. The ICR argues that the project supported, through its facilities, training, and equipment, many of these achievements, although it does not indicate the type of training or equipment that was provided.

The country also experienced a decline in the number of malaria cases, from 1,273 cases in children in 2004 compared to 62 cases in 2012. The ICR argues that while other partners were active in this area, the project contributed importantly to this outcome through its support to the Ministry of Health (the National Center for Endemic Diseases) and its overall support to rural health infrastructure (ICR, page 14). Likewise, the percentage of pregnant women with HIV/AIDS dropped from around 5 percent in 1996 to 1.5 percent in 2012. It should be noted that knowledge of HIV/AIDS did increase substantially during the project period, and the project's support probably contributed to this result both through its direct support and its financing of NGOs (ICR, page 14).

Improved local governance for basic health services is rated Negligible, due to a lack of evidence on both outputs and outcomes.

Outputs

The project financed support to local NGOs to implement activities in the health sector. In addition, the project provided support for the Ministry of Health, although this is not likely to have had much impact on local governance. The ICR does not report any other activities related to improved local governance. According to the results framework, only 30 percent of the planned financing for community-driven initiatives was disbursed, although the ICR does not explain why this was the case. 95 percent of planned sub-projects were successfully completed, although the ICR does not define "successful completion" or clarify the distribution of sub-projects between the education and health sectors.

Outcomes

The ICR does not report any indicators or evidence of increased local governance in the health sector, nor is there any reason to believe the project's activities would have led to increased local governance.

5. Efficiency:


Efficiency is rated Substantial.

Overall, the decision to merge two sectors in one project produced administrative synergies, which are important in a country with a small population and limited capacity. As is common in other projects in Sao Tome and Principe, the market for procuring goods and services is quite limited and given the small size of the project, is not attractive to outside contractors (ICR,page 16).

The portion of the project focusing on education was largely implemented on schedule, within the expected budget. The use of NGOs helped reduce the cost of implementing activities. The project was also able to work with other partners in the sector (for example, the Gulbenkian Foundation) to provide necessary inputs for the sector.

The health components of the project were delayed, in part due to the small size of the market and the lack of bidders (ICR, page 16). The project was extended to allow the health components to complete activities and to provide time for additional financing.

The total rate of return for the project at appraisal is taken from the estimated rate of return for two different sub-components (focusing on malaria and HIV/AIDS), weighted by their proportion in Component 3. Component 3 accounts for 7 percent of total financing. Given the complex nature of the project, with different components in different areas, it is difficult to calculate a meaningful single rate of return.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
57%
7%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


The relevance of the project's objectives is rated high, reflecting the continued importance of the project's activities to the Government and the Bank. The relevance of design is rated modest, reflecting a reasonable causal chain for the objectives related to access and quality but shortcomings due to lack of planned activities dealing with improved local governance in the health and education sectors. In the education sector, achievement of objectives is rated high for improved access, substantial for improved quality, and negligible for improved local governance. These ratings reflect major gains in enrollments and important improvements in quality, but a lack of information on local governance. In the health sector, the achievement of increased access and quality are both rated substantial. The country saw increased access to a baskc package of health services and important gains on a number of health outcomes, and it appears that the project contributed meaningfully to these results. As with education, the objective of improved local governance in health is rated negligible due to a lack of evidence. Efficiency is rated substantial, as the project worked reasonably well and delivered most of its outputs in a timely, cost-effective fashion.

Taken together, these ratings are indicative of moderate shortcomings in preparation and implementation and therefore an Outcome rating of Moderately Satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:


Sao Tome and Principe's economy is growing rapidly, giving the government resources to sustain the project's investments. In addition, the country has received strong commitment from other development partners, particularly in the health sector (ICR, page 11). The government's development strategy proposes sustaining the project's approach moving forward, with a focus on improving the coverage and quality of health and education services.

The project supported many structural reforms, particularly in education, such as transferring grade 6 to the primary level and guaranteeing promotion from grade 1 to grade 2. These reforms are not easily reversed and show commitment on the part of the government to implementing these major reforms.


a. Risk to Development Outcome Rating: Negligible to Low

8. Assessment of Bank Performance:

a. Quality at entry:

Sao Tome and Principe is a small island country, which largely drove the project design to focus on two sectors instead of concentrating on one. The country's capacity to administer multiple projects is limited, and the Bank's capacity to supervise a number of smaller projects is also limited. Given the needs in both sectors, the decision to prepare a multi-sector project was sound. The discussion of the alternatives considered and rejected captures well the various options that the project team faced working in a small economy (PAD, pages 7-8; ICR, page 7).

The appraisal had a thorough review of fiduciary risks, which were rated high due to the country's lack of experience with Bank procedures. The Bank's approach to financial management (PAD, pages 44-51) was complex but did address the fundamental risks effectively. The Bank's approach to procurement was also realistic, encouraging the bundling of goods and civil works to make the project more attractive to internal bidders. The appraisal also supported the development of two safeguard plans, focusing on the environment and resettlement. These plans were adequate for the project.

On the technical side, the Bank conducted a sound analysis of the problems facing the country and did a good job in collecting and analyzing the data that were available. This is particularly clear for the health sector in Annex 1 (Country and Sector Background) of the PAD as well as for both sectors in Annex 9 (Economic Analysis) of the PAD.

The project was subject to a Quality Enhancement Review in January 2004. This review highlighted the likely challenges that the project would face and provided concrete recommendations. The ICR reports that these recommendations were appropriately incorporated in the PAD (ICR, page 8).

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:
Despite the concerns at entry, the ICR (page 8) reports that implementation was relatively smooth and the project always maintained a positive rating. The Bank conducted an average of two supervision missions per year. There was only one task team leader and a relatively stable team throughout the life of the project (ICR pages 17-18). During the mid-term review (November 2007), the team reviewed the validity of the PDOs and determined that they remained appropriate (ICR, page 9). This was a shortcoming, as the project appears to have done little to support local governance in either the health or education sectors. The Bank agreed to an extension to complete civil works (one year) and later to accommodate additional financing (two additional years), which appear to have been sound decisions.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The government provided necessary support during the development of the project, including the disbursement of budget as required as well as provision of complementary resources to support the project's objectives. According to the ICR (page 18), the government developed increasing ownership of the project during implementation. Changes in ministries and other key staff led to delays in certain aspects of implementation (ICR, page 18).

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

The government project administration agency (Project Administrative and Fiduciary Agency, AFAP) was responsible for the implementation of the project, working through the Ministry of Health and the Ministry of Education, Culture and Training, as well with the Federation of NGOs (FONG).

The ICR does not provide much direct evidence on the performance of these organizations. However, it does indicate that "AFAP's role in project implementation went far beyond fiduciary management" and that it provided support to all aspects of project implementation. The ICR also indicates that AFAP provided stability during changes in the government (page 19). Furthermore, the ICR states that "supervision missions consistently found that financial management and procurement were satisfactory and continually improving" (page 10).

There was a significant increase in financing for project administration. The planned amount was US$ 0.7 million and the actual amount was US$ 1.5 million. Part of this increase was likely due to the extension of the project and implementation of additional financing, although the ICR does not explain this increase.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The project had a reasonable and well designed results framework, outlined in the PAD. The only weakness in the results framework was the lack of indicators on local governance. Although two indicators were included in the results framework, they do not capture the project's impact on local governance of basic services. As much as possible, the M&E system relied on commonly used indicators and externally produced surveys (ICR, page 9).

The project design detailed how and when data were to be collected. Most of the indicators had recent and monitorable baselines. The project supported the contracting of M&E specialists in both AFAP and in the FONG to assist these two organizations with collecting and managing information (PAD, pages 10-11).

b. M&E Implementation:

One M&E officer worked throughout the life of the project, which led AFAP to develop strong M&E capacity. Project activities were continually monitored, and the project also collected data for the results framework as required. The project was proactive in updating indicators as required.

a. M&E Utilization:

The ICR (page 10) indicates that the project's M&E data and analysis were used by the government. The project's health indicators were among those used to develop government strategies and the Government's Poverty Reduction Strategy Paper. Data on project indicators were also regularly used in the preparation of annual work plans (ICR, page 10).

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:

The project triggered the environmental assessment safeguard (OP 4.01) due to the risk of medical waste. According to the ICR (page 10), the project developed and implemented successfully a national health care waste management plan. The project also triggered the resettlement safeguard (OP 4.12) and carried out the necessary baseline studies on resettlement. Compliance with safeguards was rated moderately satisfactory throughout the life of the project (ICR, page 10).

b. Fiduciary Compliance:

The project was originally rated as having high fiduciary risk. The ICR (page 10) reports that there were no major issues with either financial management or procurement, and the overall ratings for these continued to improve during the life of the project. The ICR does not report on the status of audit reports.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Negligible to Low
The project's contributions seem to be at little risk, given the Government's endorsement of its strategy, the growing availability of resources, and continued support from a number of development partners.  
Bank Performance:
Satisfactory
Moderately Satisfactory
The Bank provided strong support in preparing the project. The ICR's evidence on supervision is incomplete.  
Borrower Performance:
Satisfactory
Moderately Satisfactory
Turnover in the Ministries led to some delays.  
Quality of ICR:
 
Unsatisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

The ICR offers several lessons (pages 18-19) regarding the preparation of multi-sectoral projects in small countries. Some of the most salient include:
    • The use of a fiduciary agency for project management is critical for successful implementation by country institutions in a low-capacity environment. In this project, the fiduciary agent went far beyond fiduciary management and acted as a technical assistant to the health and education ministries in various aspects of project execution, particularly by supporting the development of terms of reference, contributing to technical discussions, and training staff on monitoring and evaluation.
    • Having one project for two or more social sectors in a small country can promote relative economies of scale for the fiduciary and procurement functions of the project, as well as for Bank supervision.
    • Very small states have particular organizational issues (for example, structure, communication, and coordination) that need to be taken into consideration in project design and assessment.

14. Assessment Recommended?

Yes
Why?
This project has several interesting features. First, it is a joint project that involved two ministries and the NGO federation as on-the-ground implementers. Second, it serves as an example of how projects can be implemented in small economies. Third, despite many initial concerns, it appears that the Government had the necessary technical and fiduciary skills to implement the project. This project may serve as a good example of the "Science of Delivery" in action.

15. Comments on Quality of ICR:

While the ICR includes a good discussion of the project preparation and design, the ICR does not present a clear picture of how the project's resources were used, particularly in the area of health. For example, there is no indication of how many health workers were trained or what, if any, equipment was purchased using the project's resources.

More critically, the ICR also does not present a full set of outcomes. Section F of the introduction (Results Framework Analysis) is blank. Although the PDO indicators are included in the text of the document (Table 1), there is no reporting of the intermediate results, which are included in the Results Framework in Annex 3 of the PAD and appear to have been tracked during supervision. The lack of information on inputs combined with lack of information on these intermediate results complicates the identification of the causal chain. In addition, the ICR provides very little information on the "improved local governance" objectives of the project. The ICR's discussion of both the Bank's and the Borrower's value-added is also limited.

The project team provided the ICR review with evidence that should have been included in the ICR. This was primarily data from the project's results framework that would ordinarily have been included in section F.

a. Quality of ICR Rating: Unsatisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions