Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Ry Urban Wtr Supply & Sanitation Apl


  
1. Project Data:   
ICR Review Date Posted:
09/14/2012   
Country:
Yemen
PROJ ID:
P057602
Appraisal
Actual
Project Name:
Ry Urban Wtr Supply & Sanitation Apl
Project Costs(US $M)
 150.00  170.60
L/C Number:
C3700
Loan/Credit (US $M)
 130.00  128.28
Sector Board:
Water
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  08/01/2002
 
 
Closing Date
12/31/2007 12/31/2010
Sector(s):
Water supply (45%), Sanitation (45%), Sub-national government administration (5%), Central government administration (5%)
Theme(s):
Pollution management and environmental health (33% - P) Access to urban services and housing (33% - P) Water resource management (17% - S) Other financial and private sector development (17% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Lance Morrell
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The project was the first in a proposed series of two Adaptable Program Loans (APL). As stated on page 3 of the Project Appraisal Document (PAD), “The APL program purpose [is]: To provide efficient and sustainable water and sanitation services in Yemen's urban areas through: i)increase in urban water supplies initially through improved operation and reduction of water losses (Phase I) and subsequently through development of new water resources including the reuse of wastewater (Phase II); ii) improvement of sector management through establishment of financially viable regional corporations with significant participation of the private sector; and iii) provision of affordable sewerage facilities that assure protection of the environment and permit the reuse of wastewater for agriculture or artificial recharge.”

The PAD (page 3), describes the project development objective as follows: “For Phase I and II: To provide efficient and sustainable water and sanitation services in Yemen's major urban areas.”

The project development objective, as stated in the Development Credit Agreement (Schedule 2, page 13), “is to assist the Borrower in providing efficient and sustainable urban water and sanitation services.”

This Review will use the statement of objectives in the PAD as it is more specific and monitorable.

The PAD (page 7) states four expected outcomes from Phase I: “First, there should be a significant increase in the availability of water and sewerage services in the project cities. Next, a sector regulator should be established. Third, there should be significant PS [private sector] participation in the management of the Sana'a facilities. Finally, the Phase I project cities should have achieved financial viability and accountability, improved implementation capacity and be eligible for on-lending arrangements in Phase II.”

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

Component 1: Rehabilitation and Improvement of Water and Waste Water Facilities (Planned: US$122 million Actual: US$147.2 million) was designed to finance the infrastructure works and equipment required to improve the water and sanitation services in the project cities. It included:

  • Civil works, goods and equipment for the rehabilitation and expansion of the water supply networks and related facilities (pumping stations, reservoirs);
  • The rehabilitation and drilling of new bore holes;
  • The rehabilitation and expansion of waste water networks, treatment plants and related facilities; and
  • Basic water supply and waste water treatment works for six small urban centers with undeveloped networks that could not qualify to participate in Phase 1 of the APL.

Component 2: Institutional Support (Planned: US$16 million Actual: US$23.4 million) was designed to provide technical assistance, training and equipment to support the implementation of the project, build the local corporations’ capacity and prepare the second Phase of the Program. The assistance was to include training and workshops for local corporations, the National Water and Sanitation Authority, and staff of the Ministry of Water and Environment. Local and international advisors would be provided for institutional development, and capacity building to establish a regulatory body for the water and waste water sector. To address the need for comprehensive investment planning for the waste water collection, treatment, sludge disposal and waste water re-use, this component also included:
  • A stand alone Sector Environmental Assessment for the Sana’a Capital Trust area that would result in: (i) a government policy on the waste water issue; and (ii) a waste water investment plan for the short, medium and long terms, and
  • A feasibility study for a new waste water treatment capacity in the Sana’a Trust area.

Component 3: Maintenance Works in Sana’a (Planned: US$10 million Actual: US$0.00) was designed to finance maintenance related works, equipment and services for the existing water and waste water facilities in the city of Sana’a.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Costs

The project’s final costs were US$170.6 million, US$20.6 million or 14% above the appraisal estimate of US$150.0 million (including contingencies). The cost increases were primarily in the physical works (water and waste water) in the large cities, where the actual costs were about US$38.4 million (40%) above appraisal estimates while the costs for the physical works in the small urban centers declined by about US$13.0 million or 52%. The cost increases expressed in dollars were due largely to the appreciation of Special Drawing Rights during the life of the project, which enabled more work than expected to be carried out.

The signing of a lease contract for the capital city (Sana'a) was a disbursement condition for Components 1 and 3 in that city. Three international companies were short-listed and participated in a pre-bid meeting in October, 2002. However, none of them submitted a bid by the deadline, citing unfavorable security conditions. Upon the Government's request and commitment to move from a lease to a management contract, the Bank agreed (in June, 2003) to unblock the funds allocated for civil works in Sana’a (US$16 million), but continued to block the funds allocated to Component 3. Therefore, Component 3 (Maintenance Works in Sana'a) was never implemented. The funds allocated for that Component (US$10.0 million) were reallocated to other activities.

Financing

US$128.28 million of the original IDA credit of US$130 million were disbursed. Due to the depreciation of the dollar against Special Drawing Rights (SDR), the dollar amount disbursed represented SDR84.67 million of the SDR104.2 million made available at Board approval. On December 8 2010, an amount of SDR19.52 million (US$29.58 million equivalent) remained unspent -- despite extensions of the closing date totaling three years -- and was canceled. SDR16.27 million (US$24,65 million equivalent) of this consisted of uncommitted funds. The remaining SDR 3.25 million (US$4.92 million equivalent) were related to the financing of misprocured consultancy services in February, 2006.

Borrower contribution

The financing provided by the Borrower more than doubled from US$20.0m at appraisal to about US$42.32m.

Dates

The closing date of the project was extended twice for a total of three years (from December 31, 2007 until December 31, 2010) to accommodate implementation delays. The reasons were:

  • A first extension of almost two years, until December 15, 2009, was granted on December 17, 2007 to complete the ongoing works in some of the project cities and to allow an opportunity to allocate usefully the excess funds (resulting from the Special Drawing Rights appreciation) to new sub-projects and other activities.
  • A second extension until December 31, 2010 was granted on August 20, 2009 in order to complete all contractual commitments under the project.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial. The project's objectives remain relevant to the Government’s December 2008 National Water Sector Strategy and Investment Program. The Government’s goal is to “increase urban water supply and sanitation coverage while keeping services affordable to the poor, safe, sustainable and properly regulated”.

The objectives are also consistent with the World Bank Group's Country Assistance Strategy (CAS) for Yemen, covering the period FY 2010-FY 2013. During this period, the Bank group intends to direct its assistance towards: (i) increasing annual water savings through improved water efficiency (mainly in irrigation); and (ii) extending rural access to water supply and sanitation services. Improving the efficiency of urban water supply is cited in the CAS (page 25) as one of the factors supporting the Fourth Strategic Objective, "Slow down the depletion of water resources."

b. Relevance of Design:
Modest.

Project design was only partially relevant in that attention was given to improving the extent and quality of water and sewerage services by rehabilitating and expanding existing networks and through improved operations. Additional water resources were only to be developed in the second phase of the program. Components 1 and 3 of the project were intended to improve the physical networks through rehabilitation and expansion of civil works and through improved maintenance of the networks in four major cities. Component 2 was to fund the technical assistance needed to improve the capacity of the operators and to establish a regulator. However, insufficient account was taken of increasing resource scarcity due to the depletion of the ground water which was the main source of supply for the project cities. In the absence of adequate supplies of water, the impact of network improvements on service quality and financial viability would be severely reduced.


4. Achievement of Objectives (Efficacy) :

The degree of achievement of the project's development objective -- to provide efficient and sustainable water and sanitation services in Yemen's major urban areas -- is assessed below under the headings of the objective's two component parts (efficiency and sustainability).
Outputs
Civil Works:
As a result of the civil works financed by the project, a total of 71,196 new or rehabilitated water and sewerage connections were provided, exceeding the appraisal target of 60,000.
Sana’a: Production – (i) a 5,000 m3 ground storage tank, and (ii) 3 boreholes, drilled (out of contracts to drill a total of 15).Distribution – (i) 47 km of secondary water pipes, (ii) 161 km of tertiary mains, and (iii) 12,547 new connections, (iv) 20,000 water meters and values were procured with 7,500 now available in inventory for future use.
Taiz. 615 km of new distribution water pipes were installed, 33,000 water meters and valves were replaced, and two reservoirs were constructed. In addition, the project added over 31 km of sewer lines including 1,312 connection chambers, and an interceptor was built to convey waste water to rehabilitated lagoons.
Al Hodeidah. 27 km of transmission pipeline were built to connect the new well field funded by the Islamic Fund to the distribution network. 19 km of distribution pipeline were installed and 12,000 connections were replaced. The sewerage network was expanded with 1,650 new connections and the rehabilitation of 1,855 manholes.
Al-Mukalla: Existing boreholes were rehabilitated and new boreholes drilled. The sewerage network was expanded, with a total of 6,700 new connections, 60 km of pipeline, and a waste water treatment plant with a capacity to process 27,000m3 twice per day.
Dhamar: Works including transmission mains, sewer trunk lines, reservoirs of 6,000 m capacities and distribution networks covering a large part of the city reached 80% achievement at the end of the Project. They were fully completed in May, 2011.
Al-Qaidah: Works for a new waste water collection and treatment system were initiated but suspended due to Contractor’s default. They should be completed by March 2012.
Aden and 14 small urban centers: Drilling of test wells completed.
Institutional Support and Capacity Building
  • The establishment of local water and sanitation corporations in Hodeida, Mukallah and Taiz was carried out before project appraisal.
  • Comprehensive technical assistance in water and sanitation reform was provided to the Ministry of Water and Environment, including a review of the Ministry's organization, methods, staff and capacity to identify major short comings. There was also technical support for the Project Management Unit (PMU) housed in the Ministry of Water and Environment.
  • A performance evaluation of the nine local corporations was carried out.
  • Technical assistance for the decentralization of six branches of the National Water and Sanitation Authority was provided.
  • A study for the establishment of a Water Sector Regulatory Authority was carried out, but the Authority has not been created.
  • Feasibility studies, environmental assessments and the preparation of tender documents for additional cities to be covered by the Program were completed.
  • A feasibility study and preparation of tender documents for a brackish water desalination plant in Taiz and a hydro-geological model for Al-Hodeidah were completed. These studies will assist the Government in planning and implementing the next phase of investments in the sector.
  • A large number of local corporations and autonomous utilities received basic training in utility management.
  • Office equipment, computer hardware and software, vehicles and machinery were provided for the PMU and for five small centers that have been corporatized during the project period.
  • Consulting services were provided for project supervision at the central level by the Ministry and by the PMU. However, since this absorbed approximately 50% of the funding from Component 2, insufficient financing for support at the local corporation or utility level was left.
Outcomes
1. To provide efficient water and sanitation services in Yemen's major urban areas. Modest.
  • Unaccounted For Water (UFW) in the four major cities was reduced from 36% to 34% in Sana'a and from 40% to 22% in Taiz. However, the rate remained unchanged at 37% in Hodeidah and rose from 32% to 36% in Mukalla. The target in all four cities was 35%.
  • The target of increasing the availability of urban water and sewerage services by at least 20% was not achieved in any of the four major urban centers. In fact, availability -- measured in terms of cubic meters of water sold per connection per day -- actually fell in all four cities over the project period. According to the ICR Data Sheet (page iii) , it is expected that, by the end of 2011, the target would be achieved in Sana’a once the wells under construction there are commissioned.
  • The frequency of water availability from the public systems for all customers was targeted to double the 2002 level. However, the 2002 level is unrecorded so there are no baseline data. The ICR Data Sheet (page iv) records frequencies at project closure of between 12 and 24 hours in Sana'a, Hodeidah, and Mukalla, and of "less than once a week" in Taiz.
  • Although the volume of water sold in all four cities rose between appraisal (2001) and 2009, employment in the four water companies rose even faster so that water sold per employee declined:
    CityWater sold (000 m3)No. of employeesWater sold per employee (m3)
    200120092001200920012009
    Sana’a10,37313,8008431,24612,30511,075
    Hodeidah 7,980 9,681400 62019,95915,615
    Mukallah 7,26714,9674491,14616,18513,060
    Taiz 4,085 4,305400 54510,213 7,899
Source: ICR, pages 27-30
  • By an alternative measure of productivity (number of employees per thousand water and sewerage connections), Sana'a and Hoedeidah improved slightly between 2001 and 2009 (from nine to eight and from six to five respectively), while Mukallah and Taiz worsened (from nine to eleven and from six to seven respectively).
  • "Significant private sector participation in the management of the Sana'a facilities" (PAD, page 7) was expected at appraisal, not as an end in itself, but rather "as an effective means to achieve the primary objectives of enhanced access, quality/reliability and efficiency" (ICR, page 6). As noted in Section 2d above, a lease contract was prepared, but none of the short-listed international firms submitted a bid. Although the Government committed itself to a management contract as a substitute for the lease arrangement, this too failed to materialize. The ICR attributes the lack of private sector interest to the deteriorating security situation in Yemen. The ICR reports (page 24) that the recruitment of small private operators was initiated in Sana'a in 2010 in three sites governed by three different Local Corporations.
  • There is little discussion and almost no evidence in the ICR concerning any improvements in efficiency resulting from the investments in new and rehabilitated sanitation infrastructure.
2. To provide sustainable water and sanitation services in Yemen's major urban areas. Modest.
  • The following table compares the cash and Operating Cost Coverage (OCC) ratios for the four major cities at appraisal (2001) and in 2009. The end-project target established at appraisal is also shown.
    City Appraisal 2001 Target end project Actual 2009
    Cash ratioOCC ratioCash ratioOCC ratioCash ratioOCC ratio
    Sana 55%0.75131%1.4964%1.25
    Hodeidah172%1.50216%2.9890%1.23
    Mukallah104%1.04200%2.5287%0.91
    Taiz 81%1.10140%1.7581%1.09
The cash ratio shows cash revenues as a percentage of cash expenditures.
The OCC ratio shows the coverage of operating costs by revenues (including receivables but excluding government subsidies).
Source: ICR, pages 27-30.
  • By project closure, no city was generating sufficient cash revenue to cover its cash expenditures. The only city to improve its OCC ratio was Sana'a, and even there the achievement was below the target. Financial viability of water and sanitation services in the cities benefiting from the project was not achieved. As the ICR (page 31) points out, “despite some improvements, most notably the increase in access to water and wastewater services, the financial performance of the utilities is still unsustainable, whether in the short-, medium- or long-term.”
  • There are four main reasons for the utilities' continued financial weakness. First, except in Sana'a, tariffs have hardly been increased since 2001. In the other three cities, average revenue per cubic meter of water sold was about 70 US cents in 2009. Even in Sana'a, it was US$1.19, which is considerably less than the average cost to consumers of alternatives to piped water -- for example, purchasing from vendors -- which varies from a minimum of US$1.40 to over US$3.00 per cubic meter. Second, except for Mukallah, collection efficiency hardly improved at all and accounts receivable were well in excess of the target. Third, the utilities' small revenue base is linked to low per capita water consumption (less than 30 liters per day in some cities) combined with what household surveys suggest to be a large number of people per connection. The low consumption may well be linked to the fact that water is scarce. Water production increased significantly during the project period only in Mukallah, where it almost doubled to 22.8 million cubic meters. In the other three cities combined, production rose by only 12.5% to 43.1 million cubic meters during the eight years ending in 2009. Fourth, building, operating and maintaining a fully integrated piped network system for such low production and consumption volumes means high costs per unit of output and of sales. As noted in Section 8a below, project design did not address the water scarcity issue in any detail, and it assumed optimistically that the investments supported by the project would lead to significant consumption increases which would spread the cost over a larger sales volume.
APL Program Objectives
Urban water production was increased, but by less than anticipated. Almost no progress was made in improving sector management through establishment of financially viable regional corporations with significant participation of the private sector. Little evidence is presented in the ICR of progress towards the provision of affordable sewerage facilities that assure protection of the environment and permit the reuse of waste water for agricultural or artificial recharge. A sector regulator has not yet been established.
The Bank and the Government decided to not proceed with the second phase of the APL, and the investments planned for phase II have been included in the Water Sector Support Project (H449 RY).

5. Efficiency:

The PAD (page 14) calculated an economic rate of return (ERR) of 24%, but which is stated to cover only US$25 million worth of investments. The ICR (Annex 3, Section 2) reports that reproducing the PAD's cost-benefit analysis in order to estimate an ERR at closure would be "fraught with difficulties" because: (i) the assumptions behind the economic analysis in the PAD are unclear; (ii) the supply of water entering the processing system declined during project implementation; (iii) in some towns, IDA-funded investments were supplemented by those financed by other external partners leading to difficulties in benefit attribution; (iv) data on benefits are very incomplete; and (v) the collection of further data was rendered almost impossible by the deterioration in the security situation.

The ICR, therefore, made what it describes as a "qualitative assessment based on unit costs, which identified that the expected economic returns from the investments funded by the Bank were affected by the depletion of the groundwater, which is the primary source of water in all the project cities" (page 11). The high unit costs reflect the fact that the project was planned without taking into account the severe constraints on water availability. The network was expanded and rehabilitated, while the water supply available for processing was declining, leading to under-usage of the infrastructure, reduced benefits and high unit costs.

The project took eight years to complete instead of the five years foreseen at appraisal. Much of the delay was the result of administrative inefficiencies.

Efficiency is assessed as negligible.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
24%
16.7%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of objectives was substantial. However, the project achieved its goal of providing efficient and sustainable water and sanitation services in Yemen's major urban areas only to a modest extent. The water supply and sewerage networks in the four major cities were expanded and rehabilitated, but the data suggest, at best, marginal improvements in efficiency as measured by reliability and frequency of service and labor productivity. The utilities remain financially unviable, and sustainability of the improvements has not, therefore, been achieved. Efficiency of the investments, rated as negligible, is severely undermined by the declining ground water supply, which was not adequately reflected in project design, and which results in under-usage of the infrastructure and high unit costs. Outcome is assessed as moderately unsatisfactory.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:


The risks to the development outcome are as follows:

Security: The difficult security situation in the country makes any investment project complex and high risk.

Institutional and Political: The legal and regulatory frameworks for water sector governance remain incomplete, and the regulatory agency has not yet been established, which increases the level of uncertainty regarding the Government’s commitment to the need to develop efficient and sustainable water and sanitation services. The Government's unwillingness or inability to date to address water usage conflicts and effectively manage water resources in the mountain region (Sana’a and Taiz) jeopardizes the sustainability of the project and prevents the affected local corporations from developing their production capacity and to satisfy current and projected demand.

Technical: The limited human resource and technical capacity of sector institutions make developing efficient utilities more difficult.

Financial: The lack of a tariff policy which enables operating cost recovery undermines the viability of all investments and service delivery operations.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:

  • Given the lack of prior progress in reforming the sector, the expectations concerning cost recovery and the participation of international private sector operators were over-ambitious. The inclusion of private sector participation in the outcomes was questioned at the preparation stage by the Regional Operations Committee on the basis that the project should rather focus on improving the operational efficiency of the utilities and on enhancing water and sewerage services. The project could have given less emphasis to "significant private sector participation" (PSP) and instead emphasized improved management practices and strengthened managerial and technical capacity.
  • In view of ambiguities with regard to the Government's long term commitment to water sector reform, the limited quality of sector dialogue, and the lack of experience in the country with Adaptable Program Loans, it is questionable if such an instrument was appropriate for this project.
  • Insufficient account was taken in project design of the implications of the diminishing availability of ground water, which was the main source of water supply for the project cities, especially in Sana'a and Taiz. In Sana’a, it was assumed that the Sana’a Basin Water Management Project (Cr. No. 3774-YEM) would help conserve water that would be available for urban uses. This did not occur. In Taiz, it was expected that if leaks were controlled and the network redesigned to work in sectors or areas of the city, then there would be better service. However, in the absence of adequate supplies of water the impact of network improvements on service quality would be severely reduced. A Social Assessment had been undertaken in November 2001, that mentioned the importance of looking into water resource development as an urgent priority, and suggested alternative options such as collecting rainwater and optimization of water use in irrigation.
  • Design did not address the need for the staffing of a project management unit prior to the project being declared effective. The delay in staffing resulted in a significant delay in project implementation.
  • Design placed too little emphasis on the importance of sound and solid corporate governance arrangements for achieving financial sustainability. Outcome indicators focused on cash flow and other accounting measures without adequate phasing and with insufficient time allocated to achieving financial sustainability.

Quality-at-Entry Rating: Unsatisfactory

b. Quality of supervision:

  • The Bank supported the project extensively through regular supervision missions despite deteriorating security conditions leading to significant restrictions on travel within the country.
  • Once the lease contract bidding process failed, the Bank’s decision not to disburse the funds allocated to maintenance works in Sana’a prevented Sana’a's Local Corporation from receiving some of the resources needed to improve its performance.
  • Institutional support planned for the local corporations did not materialize to the extent needed, as more than half of the financing under Component 2 was consumed at the central level by the Ministry of Water and Environment and the PMU, while very little was done to improve the situation at the local level. Greater proactivity at an earlier stage of implementation could have led to the reallocation of funds, which were later cancelled, to the providision of the technical assistance and capacity building needed by the local corporations.
  • Indicators were not adjusted to address the difficulty in measuring them or to make them more realistic. The mid-term review -- which was carried out in February 2006 while 90% of the credit was committed but only 22% of the funds had been disbursed -- was too optimistic when it concluded that the project was “on track” to achieve its objectives. The opportunity to review and restructure the project was not taken.
  • Since the rating for the Development Objective was downgraded to Moderately Unsatisfactory shortly after the third extension of the credit closing date was granted, it is questionable whether the Bank should have agreed to process the requested extension.
  • It is unclear if weaknesses identified in environmental management and mitigation of environmental damage during the implementation of project-supported activities were addressed (see Section 11 below).
  • An allegation of inadequately compensated land acquisition -- which preceded the project but which led to the breakage of sewage pipes financed by the project, and consequently to raw sewage being used for food crop irrigation -- remained unresolved at project closure (see Section 11 below).
  • There was non-compliance with Bank procurement guidelines throughout project implementation (see Section 11 below).

  • Quality of Supervision Rating: Moderately Unsatisfactory

    Overall Bank Performance Rating: Unsatisfactory

    9. Assessment of Borrower Performance:

    a. Government Performance:
    Central Government, in particular the Ministry of Water and Environment and the Technical Secretariat, provided support to move the project forward at difficult junctures. For example, the Government kept on attempting to push the private sector participation agenda forward after the failure of the lease contract. The Minister of Water involved himself personally in trying to resolve deadlocks with the contractors in Mukallah. Strong government commitment was also shown by the fact that the Borrower's financial contribution to the project was more than double appraisal estimates.

    Tardiness in hiring staff for the Project Management Unit (PMU), including the Project Director, contributed to delays at the beginning of implementation. The Project Director was hired on April 26, 2003 and the Procurement Specialist in June 2004 (6 months and 19 months respectively after the project had been declared effective). In the meantime, the project had to rely on the PMU in place for the then ongoing Sana'a Water Supply and Sanitation Project, but the personnel there had little incentive to deal with a procurement plan that included more than 50 packages, and for which they were under-staffed. As a result, the first pre-qualification evaluation reports and tender documents were received only one year later, although the Bank had already granted its non-objection to most of the contractors' pre-qualification documents and some drafts of tender documents during appraisal.

    In Mukalla, land acquisition problems and the issuance of a new urban development plan required the relocation of the waste water treatment plant a few weeks after contract award. This resulted in extensive delays, a need for new designs, and conflicts with the contractors which could have been avoided with better coordination and planning.

    Government Performance Rating: Moderately Satisfactory

    b. Implementing Agency Performance:

    The performance of the central PMU, housed in the Ministry of Water and Environment, was adversely affected by inadequate staffing arrangements. Procurement activities did not meet the standards required for a Bank funded project. Inefficient procurement and non-compliance with Bank procurement guidelines persisted throughout the life of the project (see Section 11 below). One case of mis-procurement resulted in a partial cancellation of the IDA credit.
    At the local level, the project implementing units set up within the local corporations proved to be ineffective, especially in Taiz and Mukallah which resulted in significant delays. The goal of transferring capacity to the local corporations was not attained, in part because insufficient financial resources were available.

    Implementing Agency Performance Rating: Unsatisfactory

    Overall Borrower Performance Rating: Moderately Unsatisfactory

    10. M&E Design, Implementation, & Utilization:

    a. M&E Design:
    The monitoring and evaluation (M&E) framework included five outcome indicators. Of these, the establishment of local water and sanitation corporations in Sana’a, Al-Hodeida, Al-Mukalla and Taiz was irrelevant as it was already met at the time of appraisal. One indicator (achievement of an equitable balance between improving water and waste water services and expanding coverage) was not adequately defined to function as a measurable indicator. Another indicator (achievement of financial viability) proved to be too ambitious.

    The design also included four output indicators. The number of connections was to be used to monitor the infrastructure component of the project. An indicator to monitor the works related to waste water treatment would also have been useful, such as the percentage of waste water treated or the quantity of biological oxygen demand removed. Unaccounted For Water (UFW) and financial ratios are standard indicators used to support policy dialogue and progress towards achieving financial viability.

    The M&E system was to be managed by the central PMU housed in the Ministry of Water and Environment.

    b. M&E Implementation:
    Although anticipated in the PAD, no baseline survey was carried out to establish the situation of the targeted beneficiaries and the status of monitoring indicators at the outset of the project. Therefore, several of the indicators could not be used as a basis for monitoring project achievements. Three indicators were consistently reported through progress reports and supervision missions: the UFW ratio, the number of additional connections provided, and the cash revenue/expense ratio.

    Faced with the difficulty of collecting accurate population data and the lack of a baseline, the 20% increase in “service availability” indicator was replaced by the volume of water provided by connection (in M3/connection/day).

    The Performance Indicators Information System PIIS) put in place by the Ministry of Water and Environment in 2006 with the support of the German Technical Cooperation Agency (Gesellschaft fr Technische Zusammenarbeit or GTZ) which is monitoring 17 performance indicators for each utility, including two of those mentioned above, could have been very useful for monitoring the project, but the value of this tool was reduced by the very late publication of the results (more than two years elapsed between data collection and reporting). At the time of the ICR preparation, data were only available for 2006 and 2007.

    a. M&E Utilization:
    Indicators were mainly used to determine the degree to which triggers were met for proceeding with Phase II (which did not occur as the investments were transferred to another project).
    The M&E framework put in place within the Government's National Water Sector Strategy and Investment Program uses some of the indicators from the project’s M&E system.
    Two further initiatives, while still incomplete, may assist policy makers in the future:

    • The PIIS would develop into a useful strategic tool if data collection and reporting could be speeded up.
    • A Performance Agreement between the City of Sana'a and the Sana’a Local Corporation was signed on January 17, 2010 by the Mayor; it contains a reasonably comprehensive set of performance objectives and indicators.

    M&E Quality Rating: Modest

    11. Other Issues:

    a. Safeguards:
    The project was rated Category "B" for purposes of environmental assessment, since it would mainly rehabilitate existing facilities. As the ICR points out (page 8), this was justified for Sana’a and Taiz, but not for Mukalla where the project was to support a new waste water treatment plant. According to the PAD (page 23), two safeguard policies would be triggered -- Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.12).
    Environment.

    • The ICR reports (page 8) that Environmental Mitigation Plans approved by IDA were developed for each of the sub-projects within the project. An Environmental Safety Officer was hired by the central Project Management Unit (PMU), but resigned in July 2005. The gap was filled by an expert mobilized by the consultant contracted to provide technical assistance to the PMU for project management. The audit carried out by this expert in October 2008, upon the Bank’s request, reported that the civil works contractors did not always have the experience and capacity to implement the environmental mitigation measures specified in their contracts. This was the case, in particular, in Al-Qaida (the contract concerned was terminated) and Taiz. The main deficiencies noted were weak safety precautions, and irregular collection of debris and waste which raised complaints from residents in some areas.
    • The Malaria Abatement Program recommended in Taiz by the Environmental Impact Assessment to mitigate the likely breeding of anopheles mosquitoes in the sewage lagoons was not implemented as the public health and malaria specialist foreseen to help the Environmental Safety Officer was never hired.
    • The ICR contains no discussion of any remedial measures taken to address the weaknesses identified in environmental management or damage caused by contractor non-compliance.
    Involuntary Resettlement.
    • The project included some limited unoccupied land acquisition in Mukallah and Taiz, which didn’t require resettlement. According to the ICR (page 8), provisions of the Land Acquisition and Resettlement Policy Framework prepared for the project were respected. However, although the Bank requested the PMU to publish the Arabic version of this document in December 2008, it was made available only toward the end of the project (March 2010).
    • In Mukalla, project implementation was delayed by land acquisition issues that had to do with tribal access rights to government lands. The courts ultimately resolved the case in the Government’s favor.
    • In Taiz, a prior environmental and social issue had a significant negative impact on the project. The community, within which the Taiz waste stabilization ponds were built before the project began, alleged that they had not been provided with the compensation (in the form of electricity and piped water connections) promised by the local government. As a result, new sewer lines built by the project were broken, untreated waste water was used for irrigating food crops, and the ponds that were rehabilitated by the project have been “reclaimed” by the local sheikh for grazing livestock. Bank supervision missions repeatedly brought this problem to the attention of the Authorities, and asked for mitigating actions. A protracted negotiation with the community began in the final years of the project, but the problem remained unresolved at project closure and raw sewage continued to be used for food crop irrigation.

    b. Fiduciary Compliance:
    The project benefitted from the preceding Sana’a Water and Sanitation Project (Cr. 3209-YE), and the PMU for that project had in place adequate staff and a sound and developed system for financial management. The project under review used both Statement of Expenses (SOE) and direct payment disbursement methods, and the ICR (page 9) reports that the flow of funds ran smoothly between the Bank and the project. The project was audited by a firm of independent auditors. While the timely submission of audit reports was an issue for some local corporations at the beginning of implementation, this was gradually corrected. The external audit report for the fiscal year ended December 2010 (end of project) was unqualified.

    Overall, procurement activities had significant deficiencies. One case of mis-procurement related to a consultancy contract resulted in a partial cancellation of funds (SDR 3.25 million, US$4.92 equivalent). In most of the civil works contracts, a completion period of 18 months was systematically specified although the actual estimated completion period was well beyond that date with intent to avoid the “price adjustment provisions” specified in the Bank’s procurement guidelines. According to the ICR (page 9), this is a generic issue in Yemen since the Government does not publish indices for construction costs.

    Minimum qualifications of bidders for works contracts did not always comply with Bank guidelines (turnover should exceed twice the annual cash flow for the contract). For instance, the tender for drilling ten bore holes in Sana’a required the bidders to have a minimum turnover of US$2 million, while it should have been at least US$6.7 million (the contract cost was estimated at US$5 million to be implemented in 18 months). Also, in Mukalla, the same joint venture (JV) was awarded two contracts while the prequalification report stated that the average annual turnover of the local joint-venture partner qualified the JV for only one contract. It appeared afterwards, that this local company was also the civil works subcontractor for Mukallah and was also selected for two contracts in Hodeidah. Clearly, the volume of work given to this JV was far in excess of their capacity, and this was reflected in poor contract performance.

    c. Unintended Impacts (positive or negative):

    The project funded several preparatory studies which have allowed other donors to contribute to the sector’s development. In particular, the project funded two contracts for 14 small cities to develop water resources studies, feasibility studies, and detailed designs. This was a major effort that was meant to prepare packages for the next phase of the APL. These studies have helped the Government implement a second stage of works with financing amounting to US$150 million funded by various regional funds (Arab Fund, Saudi Fund, Abu Dhabi Fund, Islamic Development Bank and Oman).

    d. Other:



    12. Ratings:

    ICR
    IEG Review
    Reason for Disagreement/Comments
    Outcome:
    Moderately Unsatisfactory
    Moderately Unsatisfactory
     
    Risk to Development Outcome:
    High
    High
     
    Bank Performance:
    Moderately Satisfactory
    Unsatisfactory
    There were significant weaknesses in Quality at Entry; design was based on unfounded assumptions concerning water availability. There was insufficient proactivity during Supervision in resolving implementation problems, and the Bank's procurement guidelines were not complied with in several instances. The extent to which weak capacity to handle environmental issues was addressed is unclear. Some social issues remained unresolved at closure. 
    Borrower Performance:
    Moderately Unsatisfactory
    Moderately Unsatisfactory
     
    Quality of ICR:
     
    Satisfactory
     
    NOTES:
    - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
    - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

    13. Lessons:

    The following lessons are taken from the ICR with some adaptation.
    • Expanding water supply networks when the quantity of water being fed into the system is declining results in under-used infrastructure, and provides only partial benefits. In the case of this project, the issue of water scarcity was given insufficient attention, and other water resource development options, such as rainwater collection, were not given full consideration. In situations of serious water scarcity (like that in Yemen), rehabilitation of networks - through replacement of pipes and installation of water meters – is likely to have limited effect as less water is flowing through the pipes.
    • To enhance service delivery, both improvements in infrastructure and institutional development (including capacity building and enhanced governance) are necessary, especially in countries with weak capacity.
    • Attracting international private operators to a country where they have not been present before can be difficult. The perception by the private sector of the security situation, the quality of governance, the reliability of the institutional framework, the project’s size, and expected financial benefits compared to the commercial investment, all require careful assessment.
    • Achieving full operating cost recovery is a frequent objective in Bank funded water sector projects, but one which is rarely achieved in practice. One of the main reasons is that cost recovery depends on the tariff policy over which Bank funded activities may have little influence. Full operating cost recovery is especially difficult to attain in the short term (that is, during the life of a typical project).
    • Declaring a project effective before all key implementation arrangements are in place is likely to exacerbate delays.

    14. Assessment Recommended?

    No

    15. Comments on Quality of ICR:

    The ICR provides relevant facts and data about the performance of the Project. The report reads well, and it provides a candid assessment of the conditions surrounding both the design and implementation of the project. However, a more complete discussion of the changes made to the project design based on the comments provided by the Operations Committee would have been helpful. No indication is provided of whether adequate measures were taken to address contractors’ non-compliance with their environmental obligations.

    a. Quality of ICR Rating: Satisfactory

    (ICRR-Rev6INV-Jun-2011)
    © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions