|1. Project Data:
ICR Review Date Posted:
|Ry-third Public Works
Project Costs(US $M)
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|General water sanitation and flood protection sector (30%), General education sector (30%), General transportation sector (20%), Health (10%), Other social services (10%)|
|Rural services and infrastructure (29% - P)
Social safety nets (29% - P)
Gender (14% - S)
Other financial and private sector development (14% - S)
Participation and civic engagement (14% - S)|
||ICR Review Coordinator:
|Toneema M. Haq
|2. Project Objectives and Components:|
a. Objectives:As per the Development Credit Agreement (DCA):
"to assist the Borrower in providing needed infrastructure to improve services and environmental conditions affecting in particular women and children and creating short term employment." (DCA, Schedule 1).
As per the Project Appraisal Document (PAD) :
"to continue supporting the Borrower in the (a) provision of needed infrastructure to improve services and environmental conditions (particularly those affecting women and children); and (b) creation of short term employment. The Project would also seek to ensure the sustainability of these measures through: (i) community involvement in project selection, preparation, and implementation; and (ii) development of local contracting and consulting firms." (PAD, p. 3).
IEG's assessment is based upon the DCA objective which this ICR Review disaggregates into the following two sub-objectives: (i) improve services and environmental conditions affecting in particular women and children); (ii) create short term employment. Sustainability is subsumed under the first project objective.
b. Were the project objectives/key associated outcome targets revised during implementation?
c. Components:There were three components:
1. Community Infrastructure (Estimated Cost: US$46.50 million, Actual Cost: US$81.75 million)
This was the largest component of the project, representing 90% of the total project cost. This component financed small-scale demand-driven (proposals submitted by local communities and local governments) civil works sub-projects in sectors such as health, education, vocational training, water, sanitation, etc. Most sub-projects were in the education and agriculture/water harvesting sectors. Vocational training/social security, sanitation, and health had the lease number of sub-projects. As for selection criteria, the PAD mentions that these works were to be in areas with "above-average unemployment rates" (p. 4) and the Implementation Completion Report (ICR) mentions that the sub-projects were selected based on criteria including labor content, investment cost per person-month of employment generated, and poverty alleviation (ICR, p. 3). The cost for this component nearly doubled due to steep fuel and construction material price increases in April 2005 (ICR, p. 6).
2. Technical Assistance/Consultant Services (Estimated Cost: US$3.50 million, Actual Cost: US$4.15 million)
This financed technical assistance and consultancy services for sub-project preparation and implementation. It was also to finance a baseline survey, a Socioeconomic Impact Assessment (which was carried out in 2007 but could not be conducted at the end of the project due to the conflict situation), annual audits, and training and workshops. During project implementation there was a reduction in the number of district councils financed for capacity building from 50 to 20, due to a request from the Ministry of Local Administration. The explanation for this request for reduction was that it followed the adoption of the National Strategy for Decentralization (ICR, p. 2).
3. Project Management (Estimated Cost: US$2.00 million, Actual Cost: US$4.37 million)
This financed the operation of the Project Management Unit (PMU) including training.The cost for this component more than doubled. The ICR mentions high inflation and increase in cost of living for the PMU which are most likely behind this cost increase.
d. Comments on Project Cost, Financing, Borrower Contribution, and DatesProject Cost. The original project cost was US$52.0 million and the actual project cost was US$90.27 million. An additional US$38.27 million was spent after an Additional Financing was approved in 04/22/2008 of US$29.8 million. Additional Financing was needed to meet the financing gap caused by continuous price increases beginning in 2005 in cement, steel, and fuel.
Financing. At appraisal, IDA committed SDR31.3 million (US$45.0 million equivalent) to the project. This was supplemented by an Additional Financing in 04/22/2008 for SDR18.8 million (US$29.8 million equivalent) bringing the total up to US$74.8 million. Due to an increase in SDR rates versus the US dollar, the actual credit value at the end of the project was US$47.34 million for the original credit plus US$28.80 million for the Additional Financing totalling up to US$76.15 million. There was no official cofinancing for the project, but the project was designed to help the Government attract additional funding from various donors. To this end, the Government raised an additional US$303 million from various donors, including the Arab Fund for Development, Saudi Fund for Development, OPEC, etc. (ICR, p. 21). According the project team, this was not official cofinancing but instead was parallel financing for other public works projects in Yemen.
Borrower Contribution. At appraisal the Government committed US$4.0 million and local communities committed US$3.0 million (including cash and in-kind contributions). This was nearly 14% of total estimated project costs. At project completion, government contributions amounted to US$13.26 million which was 15% of total actual project costs.
Dates. The project was restructured twice and the closing date was extended twice. None of these went for formal Board approval. The first restructuring on 07/19/2007 reduced the number of districts from 50 to 20 for training under Component 2 of the project (see Section 2c above). The second restructuring of 04/22/2008 took place when the Additional Financing was approved. The PDO was not changed but indicator targets were adjusted to incorporate results expected from the Additional Financing. This Additional Financing was requested by the Government to meet the financing gap caused by price increases in cement, steel and fuel after 2005. The closing date was extended twice, first in 05/05/2009 to enable completion of project activities particularly those related to the Additional Financing, and second in 06/22/2011 to deal with the unrest in 2011. The project's closing date was extended to June 30,2012 in order to complete all sub-projects, which got delayed due to unrest in 2011.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:At the time of project closing, the relevant country strategy for Yemen was the Country Assistance Strategy (CAS) for FY2010-2013. The objectives of the Third Public Works Project fit well under the thematic areas of this CAS. The first objective, provision of needed infrastructure, fits under Thematic Area 1.3: Enhance Infrastructure Services (CAS, p. 21). The third objective, sustainability through community involvement, fits under Thematic Area 3.3: Implement Community Level Development Actions (CAS, p. 24). The ICR emphasizes the community involvement in this project which is mentioned in the country strategy. The project's objectives are relevant to the country and Bank's prevailing strategy.
Relevance of Objectives is Substantial.
b. Relevance of Design:
The design incorporated lesson learned from the previous two projects, Public Works Project I (C2878) and Public Works Project II (C3168) . PWP I was rated Satisfactory in its overall outcome in the ICR. PWP II was rated Highly Satisfactory in its overall outcome in the ICR and Highly Satisfactory in the subsequent IEG review of the ICR. Lessons learned from these previous two projects were incorporated into project design and include: (i) principles used in allocation of sub-projects (50 poverty index, 30% population density, 20% remoteness and underserved areas); (ii) gathering baseline data on performance indictors up front; (iii) ensuring sustainability prior to sub-project selection; (iv) confirmation of operations and maintenance from relevant line ministries; (v) strict screening procedures for environmental and social safeguards prior to sub-project selection. It is not clear from the ICR if these lessons were derived from having reached desired project outcomes.
The most important feature was the demand driven approach resulting in the communities' direct involvement in sub-project selection, preparation, and implementation (ICR, p. 8). Moreover, since the average cost of each sub-project was US$110,000, local small and medium-sized enterprises could be developed. This helped to ensure sustainability. Strengthening local industry would logically lead to sustainability after project closing since there would be less dependence on foreign firms for sub-project implementation.
As for the first objective, providing needed infrastructure to improve environmental and social conditions, the project funded sub-projects to provide infrastructure. The demand-driven (by communities) approach would ensure that these were "needed" by the community. One of the criteria for sub-project selection was that they should have a high labor content. This would contribute to the achievement of the second objective; creation of short term employment.
Possible shortcomings in relevance of design include uncertainty as to whether the demand driven approach in selecting sub-projects would avoid elite capture. Additionally, it is unclear whether merging the objective of short term labor creation with the selection of social infrastructure projects could lead to an optimal selection of projects and use of resources.
Relevance of Design is Substantial.
|4. Achievement of Objectives (Efficacy) :|
Achievement of Objective 1: provision of needed infrastructure to improve services and environmental conditions (particularly those affecting women and children) is rated Modest.
1. By 2012 378 schools were built, slightly below the revised target set in 2008 of 384. It is unclear from the ICR what type of schools were built (primary or secondary).
2. By 2012 20 health centers were built, less than the target set in 2004 of 45. The mid-term review of 2007 revised the target to 15 health centers to accommodate the steep increase in the price of fuel and construction materials (ICR, p. 6). The ICR reports "number of health centers built" (p. v) but adds a caveat that the indicator measures extension or rehabilitation of existing facilities No new health centers were built since there were not enough financial and human resources to operate the facilities (see Section 7 below).
3. By 2012 ten Vocational Training Centers were built from a baseline of 6. This met the target.
4. The number of water supply schemes (47 by 2012) did not meet the target set in 2004 of 90. The mid-term review of 2007 revised the target to 50 water supply schemes to accommodate the steep increase in the price of fuel and construction materials (ICR, p. 6).
5. The number of sanitation schemes (18 by 2012) nearly met the target set in 2004 of 20.
6. The number of water harvesting schemes (187 by 2012) exceeded the revised target set in 2008 of 170.
7. There were 56 roads/paving schemes. The target for this was zero since this core indicator was added for the ICR and was not part of the PAD or Additional Financing.
1. The enrollment of students in schools in the project targeted areas increased from a baseline in 2003 of 148,000 to 256,359 in 2012, exceeding by 28% the revised target of 200,500. Although more males (134,333) than females (122,026) enrolled, the project improved gender parity since enrollment increased by 58% for females against 19% for males (ICR, p. 9).
2. The number of beneficiaries using health centers in the project areas increased from a baseline of 282,900 in 2003 to 537,347 in 2012, 29% less than the revised target set in 2008 of 760,000. This was disaggregated by females (260,985) and males (276,362) though the Socio-economic Impact Assessment of 2007 indicated that 22% of the activity of the health centers was family planning which is logically more beneficial to female health (ICR, p.9).
3. The number of people graduating from Vocational Training Centers in the project target areas increased from a baseline of 500 (achieved by Public Works Project II) in 2003 to 5,434 in 2012, 112% higher than the target set in 2008 of 2,560. Eight out of ten centers are for women and females graduated more than males (4,347 versus 1,087) (ICR, p 49). It is unclear from the ICR if this series of projects (Public Works 1-3) was the only one building vocational training centers.
4. The number of people benefitting from water supply systems in the project targeted areas increased from a baseline of 40,700 in 2003 to 209,762 in 202, 29% less than the revised target set in 2008 of 295,000. This is because fewer sub-projects were approved due to inadequate water sources making this unsustainable (see Section 7 below). The ICR noted that women likely benefitted from water supply schemes since females normally spend 2-4 hours to fetch water (ICR, p. 10).
5. The number of people benefitting from sanitation schemes in the project targeted areas grew from a baseline of 147,300 in 2003 to 197,127 in 2012, 44% less than the revised target set in 2008 of 350,000. This decreased the risk of infection from water pollution though there were more male beneficiaries than female (116,545 versus 81,455). This was because improved public toilets are used more by men since they tend to be more economically active outside the home (ICR, p. 9).
6. Number of farmers benefitting in the project areas from water harvesting schemes grew from a baseline in 2003 of 38,000 to 455,770 in 2012, 42% higher than the revised target set in 2008 of 320,000. There was no gender breakdown for this indicator, though women likely benefitted from time savings to fetch water (ICR, p. 11).
7. The number of people with access to all-season roads was added as an indicator in the ICR in order to include beneficiaries from road rehabilitation and construction. Since this indicator was not in the PAD or the Additional Financing, it is not possible to measure an improvement from a baseline, a weakness in the monitoring and evaluation of the project. The total number of people with access to an all season road was 503,249, though the ICR does not explain how many of these were already served at the baseline and how many were direct beneficiaries of the project.
8. The number of direct project beneficiaries went from a baseline set in 2003 (at the end of Second Public Works) of nearly one million, to nearly 2.2 million, exceeding by 12% the revised target set in 2008 of 1.93 million. The ICR does not explain how this number is calculated and it is unclear if and how this project continued to serve the one million beneficiaries of the previous project.
9. Altogether, 81% of all project investments were aimed at benefitting women or children directly. Thus, 29% of investments (water supply, water harvesting, and vocational training) "directly or predominantly benefit women" (ICR, p. 11). Another 52% of investments (education sub-projects) "directly benefit children and contribute to gender parity" (ICR, p. 11) though a direct link is not spelled out in the ICR.
Despite these achievements, all of which at best can be described as being at the level of early/ intermediary outcomes, more outcome indicators could have been added, such as children attending school (as opposed to enrolled in school) and reduction in the incidence of illness. These outcomes are discussed in the ICR but there are no indicators to monitor and report on them. A Socio-Economic Impact Assessment conducted in late 2006 after the Mid-term review did have some promising data. The assessment found a 25% increase in children immunized against six targeted childhood diseases, an increase of 50% of pregnant women immunized against tetanus, and an increase of 20% of deliveries occurring in health centers under medical supervision. Access to services was found to be improving with a decrease in average distance before and after the relevant sub-projects. The average distance from school to village decreased from 2.9 km. to 2.4 km.; average distance from a health facility to the village decreased from 65 km. to 30 km.; average distance from a water facility to the village decreased from 3.2 km. to 0.5 km.; and average distance between a source of drinking water and where sewerage water is collected increased from 450 meters to 2,000 meters. The Project Team credits this last indicator as minimizing risks of infection from water pollution (malaria reduced by 33.3%, worms reduced by 36%, diarrhea reduced by 16.4%, and skin diseases reduced by 16.6%). Although an final Socio-Economic Impact Assessment was to be conducted at the end of the project, it could not be due to the conflict. The Project Team maintains that "the types of interventions and communities targeted stayed the same, the findings from the 2007 SEA remain relevant and indicative of the overall outcomes as observed on the ground and confirmed by the project monitoring system."
Sustainability is included as part of this objective:
1. Operations and maintenance cost for sub-projects were affordable for targeted beneficiaries, ensuring sustainability since communities evaluated sub-project proposals (ICR, p.12).
2. The technical assistance component of the project included training of local communities and beneficiaries in the operation and maintenance of the infrastructure created (ICR, p. 3).
3. The Steering Committee whose main responsibility is approving the yearly and quarterly investment plans prepared by the project have representatives from the relevant sector ministries of the sub-projects.
4. Road sub-projects are nearly maintenance free due to the paving construction technique used (ICR, p. 8).
5. Rainwater harvesting sub-projects were designed for ease of maintenance by using slow sand filters, sedimentation traps, and public taps for water collection (ICR, p. 60).
Achievement of Objective 2: creation of short term employment is rated Substantial.
There were 118,000 person months of employment created by 2012, exceeding the revised target set in 2008 of 101,000. The original target set in 2004 was 90,000. At the end of the previous project, Second Public Works (Credit 3168), 91,000 person months of employment had been created. The goal of creation of short term employment was exceeded by 17%.
It should be noted that using person-months as a performance indicator can lead to paradoxical results. More person-months than planned seems at first sight to be a good result, but it could also imply inefficiencies if more people were used than necessary to build the infrastructure.
Since the project financed demand-driven sub-projects, there was no overall Economic Rate of Return (ERR) calculated at appraisal. There was a cost-effectiveness ratio calculated of 0.42. The benefit to cost ratio of the public works program was 1.35 (benefits exceed costs by 35%) (PAD, p. 41). The assumptions for the cost-effectiveness ratio were discussed and the calculations were shown. There was also a discussion of direct and indirect transfer benefits to the poor. There were no ERR targets for sub-projects set at project appraisal.
There were overall ERRs calculated for classrooms (17.2%) and water harvesting schemes (108%). The education sub-projects comprised 52% of total project cost. Water harvesting and agriculture sub-projects comprised 23% of total project costs. These two ERRs cover 75% of project costs and average out to an ERR of 33.8%. The main assumptions and cost benefit analysis for both education and water harvesting sub-projects were discussed. A sensitivity analysis was carried out for the water harvesting sub-projects.
The project was delayed by three years due in part to the 2011 unrest and to ensure completion of project activities related to the Additional Financing.
Efficiency is Rated Substantial.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated