Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Lb - Ba'albeck Water And Wastewater Project


  
1. Project Data:   
ICR Review Date Posted:
12/09/2013   
Country:
Lebanon
PROJ ID:
P074042
Appraisal
Actual
Project Name:
Lb - Ba'albeck Water And Wastewater Project
Project Costs(US $M)
 49.20  47.86
L/C Number:
L7117
Loan/Credit (US $M)
 43.10  43.28
Sector Board:
Water
Cofinancing (US $M)
 6.10  4.58
Cofinanciers:
Board Approval Date
  06/04/2002
 
 
Closing Date
12/31/2007 06/15/2012
Sector(s):
Sewerage (49%), Water supply (42%), Sub-national government administration (7%), Central government administration (2%)
Theme(s):
Rural services and infrastructure (29% - P) Pollution management and environmental health (29% - P) Infrastructure services for private sector development (28% - P) Other financial and private sector development (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Gillian M. Perkins
Soniya Carvalho Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
-- The objectives of the project, as stated in the Loan Agreement and the Project Appraisal Document (PAD), were to "(i) develop and strengthen the institutional capacity of the Ba'albeck Hermel Water and Irrigation Authority (BHWIA), the Zahle Water Authority (ZWA) and the Chamsene Water Authority (CWA); (ii) improve the access of customers of BHWIA to satisfactory water supply and wastewater services; (iii) involve the private sector in the operation and maintenance of the water and wastewater facilities; and (iv) rationalize the use of water through the introduction of water meters."

-- The project's outcome indicators were formally revised in 2009. Three new indicators were added: the volume of wastewater collected and treated; establishment of a water tariff based on the volume of consumption; and the hiring and training of key BRWA staff. One of the original indicators -- reduction in unaccounted for water -- was dropped, due to measurement difficulties. One intermediate outcome indicator -- increase in the number of subscriptions to the water network -- was elevated to a development outcome indicator. The other outcome indicators -- improvement in collection rate; populations served; number of water meters installed; and service contracts implemented -- and their targets were unchanged. There was no change to the project's development objectives.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1. Institutional Development (estimated at appraisal US$4.55 million; actual costs US$2.46 million)
through a Technical Support Unit (TSU) which would build institutional capacity and manage training activities across the three water authorities, supervise the timely implementation of Technical Assistance activities, and assist in developing more reliable technical and financial information systems. Training of officials, engineers, and legal staff would aim at improving capabilities of the water authorities to design, develop and implement public-private partnerships in the water and wastewater sectors, increase operational efficiency and reduce unaccounted for water and other system losses.
2. Improving and Increasing Service Coverage for Water Supply (estimated at appraisal US$14.91 million; actual costs US$17.69 million)
To be executed in two phases, this component would extend the existing water supply and distribution network in the Baa'albeck urban area and 34 surrounding villages by adding a further 83 km of distribution mains and connecting 16,000 houses, thereby providing close to 100 percent coverage of the area.
3. Improving and Increasing Service Coverage for Wastewater Collection and Treatment (estimated at appraisal US$18.65; actual costs US$12.16 million)
Complementing the existing collection systems and wastewater treatment plant, this component would (i) construct a sewerage network with a total length of 160 km of pipeline in Ba'albeck city and four surrounding villages; (ii) carry out additional works necessary for proper operation of the treatment plant including effluent storage, to provide treated effluents and sludge suitable for agricultural reuse; and (iii) finance land expropriation affecting about 100-150 plots.
4. Service Contracts (estimated at appraisal US$7.44 million; actual costs US$8.16 million)
This component would finance two service contracts: (i) for operation and maintenance of the sewerage network and all water facilities such as transmission pipelines, boreholes, pumping and chlorination stations; and (ii) operations and maintenance of the wastewater treatment plant and implementation of the wastewater quality monitoring programme.
5. Consulting Services (estimated at appraisal US$3.66 million, actual costs US$3.97 million)
Services to be financed include: preparation of tender documents; construction supervision; environmental monitoring; and studies in the wastewater sector including updating the master plan and the preparation of development programs for water supply and wastewater services in the Beka'a region.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
-- Actual total project costs amounted to 97 percent of the appraisal estimates, with the IBRD loan almost fully disbursed and the Borrower contribution at 75 percent of the originally planned US$6.10 million.
-- Actual costs by component, above, are the latest estimates provided in the ICR and amount to some $3 million less than final total costs.
-- The loan became effective on July 31, 2003, almost 14 months after approval. Disbursements were negligible until 2005. The loan closing date was extended four times -- in 2007, 2008, 2009 and 2011 -- adding a total of four and a half years to the project implementation period. Major factors causing delay were: (i) time taken for approval of the project by Parliamentary committees before the loan could become effective; (ii) restructuring of the major works contracts; (iii) the impact of the 2006 war in Lebanon, which forced a temporary halt to investments; and (iv) unanticipated constraints to increasing water subscriptions when the works were completed. Restructuring of the project's development outcome indicators was approved by the Board in December 2009, along with the extension of the closing date.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High. The objectives were and remain relevant to the Bank's country assistance strategy and to the government's development priorities. The third pillar of the Bank's 2005 Country Assistance Strategy and its 2007 Interim Strategy Note, supported resource and environmental management and included improvement of urban and rural water delivery and strengthening the nascent water authorities. Most recently, the Country Partnership Strategy (CPS) for 2011-2016 highlights the importance of improving effectiveness and efficiency in the supply of water for reducing poverty and for enabling broad-based growth (pp. 11 and 18) and proposes continued support to the sector through investment and support to an "Integrated Water Sector Policy and Strategy." In March 2012, the Ministry of Energy and Water formally adopted the National Water Sector Strategy with detailed plans for nationwide reforms to improve water, sanitation and irrigation services. The project objectives are aligned with this strategy. The targeted region of Ba'albeck was and is one of the poorest in the country with particularly pressing needs for improvement in water and wastewater services.

b. Relevance of Design:
Modest. Project design was substantially relevant to the objectives with respect to the infrastructure requirements but did not adequately address social, institutional and political factors. The proposed institutional and financing reforms were in line with current practice in the sector; but the project design underestimated the time and supporting activities needed to implement these reforms successfully in challenging political and social conditions.


4. Achievement of Objectives (Efficacy) :

Develop and strengthen the institutional capacity of the Ba'albeck Hermel Water and Irrigation Authority (BHWIA), the Zahle Water Authority (ZWA) and the Chamsene Water Authority (CWA) [subsequently merged into the Beka'a Regional Water Authority (BRWA)]: Modest
  • The project achieved many of its planned outputs relevant to this objective, although with significant delay in some cases: (i) in 2005, by-laws were approved for the regional water authorities, designed to merge several small regional water authorities in the new Beka'a Regional Water Authority (BRWA), clarify administrative policies and improve operation; (ii) a business plan was developed to improve management and financial performance, and a follow up workshop was carried out; (iii) the Technical Service Unit was established with key consultancies, software and equipment, and provided extensive support to the BRWA; (iv) a full time, permanent Director General was appointed in late 2009, after long delay; and the BRWA was finally authorized and funded in 2012 to recruit and train 102 skilled and diverse staff, including bill collectors and irrigation maintenance engineers, although positions were still not filled at the time of project closing. A variety of trainings were carried out to develop the professional capacity of BRWA management and staff, in financial planning, leadership building, human resource management and computer training, as well as a financial training program under supervision of the Ministry of Finance.
  • These outputs were insufficient to achieve the intended outcomes, although some improvements were evident at the very end of the project, under the new BWRA management. Gains in the efficiency or financial sustainability of water supply in the region were not accomplished over the decade of project implementation. The strengthened institutional capacity of BRWA was expected to be demonstrated through substantial reductions in unaccounted for water and improvements in tariff collection, neither of which was achieved. Unaccounted for water is reported in the ICR to have remained at 50 percent from appraisal until the end of 2010, falling to 46 percent by the end of 2011 (target was 30 percent), although the reason given for dropping this indicator in the 2009 restructuring was inability of the BRWA to measure it accurately. The tariff collection rate declined from 35 percent at appraisal to only 24 percent by the end of 2010, recovering to 32 percent by the end of 2011 (target was 60 percent). The utility remained unsustainable, as a self-standing business, on both the water and wastewater sides. Information is not provided on the effectiveness of the training activities conducted or on relevant changes or improvements in specific practices or processes.
  • Several factors contributed to the continuing weak performance of BRWA until the very end of the project: (i) the long delay in appointing a permanent leader - Director General - for the BRWA; failure of the government to follow through on the tariff reform to which it was committed, which could have encouraged conservation through metering and strengthened financial sustainability; (iii) neglect until late in the project of public communications and the issues and incentives facing water consumers.
  • Although the project failed to meet its set targets by the time of closing, there is evidence since 2010 of at least modest impact in strengthening institutional capacity and of momentum towards achievement of this objective. Under the new manager, public communication efforts were stepped up and the accelerating pace of new subscriptions was maintained after project closing: the project team reported an additional 536 subscriptions in the nine months from June 2012. The national sector strategy is now approved and the relevant law, including volumetric tariffs, is with the Council of Ministers.
Improve the access of customers of BHWIA to satisfactory water supply and wastewater services:
Modest
  • The project exceeded its infrastructure target for water supply, adding 134.5 km of pipeline to the new network compared with the original plan for 83 km. This not only completed the network that was started under the preceding project, as envisaged at appraisal, but extended it to new neighborhoods. The new network thus allows for connections to 25,664 commercial and residential units in 45 localities across the Ba'albeck - Nabi Chit districts, compared with only 16,000 estimated at appraisal. An estimated 93 percent of the population of the expanded project areas has potential access to the new network with meter boxes installed inside the property line. However, the number of actual connections and the number of subscriptions grew slowly and failed to match the potential of the network, despite strong efforts towards the end of the project. Based on utility records (but without benefit of a unit-by-unit census to determine actual connection rates) the ICR team estimated that the water service was reaching 53 percent of the population at loan closing, from a baseline of 41 percent at appraisal (the target was 96 percent). These figures are open to question, however, due to the extension of project activities to additional communities, inconsistencies in the population estimates, and uncertainty concerning the number of unregistered connections and connections to the old network: only after recently, after closing of the project, the BRWA began a connection-by-connection audit of the dual system, to ensure that subscribers were on the new network and notifying households on the old network that this would not be maintained. In summary: potential access (through the installation of pipes and meter boxes within property boundaries) has been provided across more neighborhoods than originally envisaged, allowing for connection to more than 25,0000 consumers; however, almost half of those with potential connections are not yet drawing water from the new system, while some 40 percent of those who are drawing water from the system are not yet paying for it. Access has thus been modestly improved, although potential access has been improved beyond the project's original expectations.
  • There were several reasons, explained in the ICR, that actual connections and subscriptions lagged behind the potential of the infrastructure. In the initial phase of construction, connections were not conditional on subscriptions and the bulk of consumers drew water without subscribing. Subsequently, the installation of meters in the meter boxes was tied to subscription but more than 60 percent of potentially-connected households chose not to subscribe. A consumer survey of 800 households conducted in March 2012 found that this was due primarily to misunderstandings on the part of consumers and to unreliability of the water supply: in many cases, consumers were not aware that the connection was available; some believed that the utility would install the service line from the meter box to the house, when this was in fact the household's responsibility; and doubts were expressed concerning water quality and the reliability of supply. Affordability ranked low among reasons for non-subscription -- only one percent indicated that network water was too expensive. Another factor discouraging some residents from making connections to the new network was that the BRWA continued to supply water through the old network, not wanting to cut this off until the new connections were functioning.
  • Part of the economic rationale of the project was the benefit to area residents of increased water consumption, estimated at the time of appraisal at 55 liters per capita per day (lcd) for households connected to the network and only 34 lcd for unconnected households. This was expected to increase though the project to 100 lcd. From BRWA reports, 5,375,720 m3 of water was consumed from the distribution system in 2011, up from an estimated 2,200,000 m3 delivered in 2002. According to the ICR economic analysis (p.43), this represents an increase in the per capita consumption of connected households to somewhere between 156 and 205 lcd, depending on the actual number of beneficiaries. In the absence of better data, the distributional impact is unclear; however, access to water in the Ba'albeck area appears to have improved substantially in terms of total quantity.
  • The project team reports that the quality of water and the service provided are much improved from the old network and superior to some other regions of the country and that the project is highly regarded by the local population. However, evidence is not provided -- such as from water quality testing, maintenance reports -- to counter the rather negative perceptions of many consumers indicated through the survey. Some 38 percent of the subscribers surveyed in 2012 said they were dissatisfied with the service, citing weak water pressure, water quality, quality of service and frequent water cuts as the main reasons. Water supply through the new distribution lines was reported to be intermittent -- typically 3-4 days of the week and for less than 24 hours in those days -- and the pressure often weak. Sixty-four percent of subscribers reported continued use of non-network water for one or more purposes, primarily for drinking or cooking. This was was attributed in the ICR to understandable concerns of consumers that water quality could be negatively affected by the cuts in supply. No baseline or follow up is available on relevant indicators and the project's achievements are therefore uncertain with respect to satisfactory water supply. The survey highlighted the need for high priority to (i) network maintenance and monitoring of water quality and (ii) communicating procedures and improvements to make sure communities were aware of them. Following the survey, the new Director General stepped up efforts to address these issues in the last year of the project. However, the ICR points out that financing the necessary improvements remains a challenge until the number of subscribers is increased and tariffs are raised.
  • The project also accomplished its planned infrastructure works for improving access to satisfactory wastewater services: the construction of a sewerage network with a total length of 160 km of pipeline in Ba'albeck city and four surrounding villages; and the additional works necessary for proper operation of the treatment plant including effluent storage. The number of individuals in the project area benefitting from connections to wastewater services increased from an estimated 18,000 at appraisal to 29,410 at project closing, reaching 31 percent of the population as estimated by BRWA (the project's target was 67 percent).
  • Plans for the Wastewater Treatment Plant, which requires a minimum level of influent for normal operations, were delayed by the pace of wastewater connections as well as the resistance of farmers (who had been irrigating local fields with untreated sewage) to diverting the wastewater to the treatment plant. The plant started operations when the sewerage network was complete in 2009 but operations were halted because the quantity of influent did not meet the minimum 2000 m3 per day needed by a plant designed for 12,000 m3 per day. Regular operations started in 2011. The ICR mission in April 2012 confirmed that the plant was by then averaging 8,000 m3 per day (exceeding the target of 5,992 m3) and that influent and effluent testing was carried out regularly. The project succeeded in providing treated effluents and sludge suitable for agricultural use. The treated effluent was chlorinated and discharged to a river and some flow was taken up on a small scale for irrigation, supporting around 1,220 hectares of farmland (with most farmers mixing the treated water with well water). The ICR also reports that the sludge digester and dewatering system were operating normally. The latest analysis by the plant manager indicated that the sludge was free of toxic materials that would prevent its use on agricultural fields; however, it was not yet being put to that use and testing was not routine.
Involve the private sector in the operation and maintenance of the water and wastewater facilities: Modest
  • This objective was achieved, in so far as a Service Operator was hired through a management contract to work with the BRWA as part of a longer term engagement that would include training and capacity building, implementation of operational management plans, and an investment fund to support performance improvements and professional development. An on-going service contract remained in operation at project closing with funding provided by the Ministry of Energy and Water. The advantages included the ability to hire the necessary skills in the context of public sector hiring constraints. No information or evidence is provided on the expected outcome (PAD p. 3) of improved efficiency through involvement of the private sector. The ICR rates the performance of the service contracts as Moderately Satisfactory.
Rationalize the use of water through the introduction of water meters:
Negligible
  • Through the installation of meter boxes and supply of meters for all connections, the project put in place the necessary physical infrastructure for the rationalization of water use through metering. However, it failed on its objective, primarily because the planned national shift to volumetric tariffs, expected at the time of appraisal, was not implemented by the government. This was piloted in another regional water authority but was not implemented nationally by the close of the project. Also, in the first phase of implementation, the BRWA's policy (subsequently corrected) was to connect all potential users and install meters, whether or not they had subscribed to the new service, with the result that many were drawing water without subscribing. After closure of the project, the needed tariff policy is still not in place and subscribers continue to pay a fixed annual rate regardless of their level of consumption. BRWA is not reading the meters as that would increase operating costs without any benefit to the utility or to consumers.

5. Efficiency:

Negligible -- Economic analysis at appraisal assessed the water supply and sanitation components separately. For the water network, it estimated a rate of return of about nine percent when the relevant investment costs under the preceding project were included and almost 70 percent when all the benefits -- of lower costs to consumers and increased water consumption -- were attributed to the new investment. The ICR reported an error in the original analysis and indicated a corrected ERR for the investment in water from 70 to 47 percent. Recalculation at the ICR stage was impeded by uncertainties in population data and unregistered connection rates. Based on the value of additional water to consumers through the new system, and the cost savings of no longer using alternative sources, the recalculation points to a negative ERR (Net Present Value of -$7.72 m. ) without accounting for the delay in benefits. Through this analysis, the major determinant of the project's economic returns was the incremental number of beneficiaries of the new service, amounting to 13,070 based on BRWA estimates at project close. Towards the end of the project, the system had increased the quantity of water delivered to consumers by two and half times, reaching about 90 percent of the appraisal projection, but only reaching about half the projected number of beneficiaries as estimated by BRWA.
-- For the wastewater system, the respective ERRs at appraisal were four percent and eight percent, depending on whether the costs of the earlier investment were included, based on the benefits of availability of treated wastewater for irrigation and of cost savings to households. but excluding health and environmental benefits. A recalculation at the end of the project, on the same basis, led to an ERR of 2.3 percent for the new investment. By the end of the project, the volume of treated wastewater production was reported to have reached 90 percent of the level projected at appraisal but, once again, the number of additional beneficiaries was smaller than projected.
-- Evidence is not available on the extent of health, social and environmental benefits from the project's investments. However, it is likely that some benefits have been realized through the increase in the number of households connected to the water network, the increase (by two and a half times) the total quantity of network water supplied to consumers, satisfactory operation of the wastewater treatment plant and improved wastewater practices of farmers.
-- It should be noted that substantial additional benefits could be achieved from the project investment in the near future at small additional cost if an effective subscription program and appropriate tariff policy are implemented. The project team reports continuing developments in this direction since project closing but no certainty on the outcome.
-- Efficiency in implementation was impacted by institutional capacity constraints and long delays. An independent procurement review in 2011 concluded that weaknesses in procurement and contract management resulted in unnecessary delays and missed deadlines (ICR p.10). However, by restructuring the procurement packages for infrastructure works, which accounted for some two thirds of project expenditures, the project succeeded in extending the water distribution network to additional locations and beneficiaries within the original budget, increasing the length of the network by some 60 percent against original plans. (This may reflect, at least in part, weakness in the original procurement plans and unit cost estimates). The project closed 4.5 years behind schedule.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
70%
50%
ICR estimate:
Yes
0%
50%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project had modest achievements on three of its objectives --- strengthening institutional capacity, improving access to satisfactory water and wastewater services, and involving the private sector in operation and maintenance -- while failing on its fourth objective to rationalize the use of water through metering. Efficiency was negligible, resulting in an overall Unsatisfactory rating.

a. Outcome Rating: Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

As noted in Section 5 above, the benefits of the project could become very substantial if the necessary policy and institutional changes can be implemented: the infrastructure is in place for metered network connections to some 93 percent of the population in the region and for wastewater connections to some two thirds, as well as for providing treated effluent and sludge for agricultural use. The likelihood of this happening is uncertain. The project team reports that BRWA performance has continued to strengthen since project closing, through water and wastewater quality testing, increases in water subscriptions, and engagement in policies for improved utility management. The national sector strategy is now approved and the relevant law, including volumetric tariffs, is with the council of ministers. The tariff policy agenda is being supported by GIZ. There remains a significant risk, however, that political and institutional constraints are not resolved and that, in the absence of sustainable financing, the new infrastructure cannot be effectively maintained and operated.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
The project was well designed to complement preceding infrastructure projects and was aligned with the government's sector reform plans of the late 1990s. However, its institutional and financial agenda was ambitious in the local context and the relevant risks were not adequately identified and accounted for. The risk of weakness in the demand for connections to the new water supply network was considered negligible by the appraisal team, despite known local factors that were highlighted in the PAD (pp. 24-25), including a high prevalence of illegal connections to the old network, the collapse of civic controls as a result of instability over several years, and distrust of government among the local population (such that the preparation team decided against a social assessment or willingness to pay survey at the appraisal stage lest it provoke a negative reaction). The willingness of farmers to change their irrigation practices was also taken for granted. As a result, essential activities throughout implementation relating to communications strategy (such as public awareness activities, household surveys, public consultations, customer relations, promotional offers) were neglected in project design. In addition, although design of the project depended fundamentally on implementation of the proposed new tariff structure, and this was noted in the PAD as a possible controversial aspect (p.30), it was not addressed as a Critical Risk (p.29) or a Critical Assumption (pp. 31-32) and no mitigation measures or contingency plans were in place. The project team reports some tension in project design between the urgent need to complete the infrastructure works of the preceding project in a quasi-emergency context and pressure through internal review to combine this with longer term policy and institutional reforms to support sustainability. Economic analysis was unsatisfactory: in particular, the ICR found that a double counting error in the analysis at appraisal led to substantial overestimation of the expected rate of return (see section 5 above).


Readiness for implementation was inadequate at the time of Board presentation. The required institutional set-up -- through merging of the three utilities into the BRWA -- was not in place. The need for tariff reform was accepted in principle but far from detailed agreement and implementation. Procurement plans and costings were found to be weak: the major works contracts were restructured at the outset of the project, delaying implementation but resulting in much lower unit costs.

Quality-at-Entry Rating: Unsatisfactory

b. Quality of supervision:
Bank supervision was intensive throughout implementation and included four missions led by the Sector Manager. The context was challenging, given the political and security obstacles in a region where mistrust of government agencies was pervasive and law enforcement difficult. Until 2009, the focus was on getting the infrastructure in place, based on the rationale that the public did not trust government efforts and could not be engaged in talking about subscriptions until the water was there. With hindsight, the project team considers this was a mistake and that social assessments should have been carried out from the outset, while the pipes were being laid. From 2009, the Bank team played a major role in promoting consumer information and communications activities and in reconciliation efforts with the farmers on improved irrigation practices. Supervision was also more proactive on the policy side, pushing a consistent message on the need for tariff reform, including conducting workshops, and engaging with other partners including GIZ who have since taken on this agenda and are testing mechanisms. The team was also successful in mobilizing funding for the construction of freshwater wells for farmers and secured additional grant funds from the Public Private Infrastructure Advisory Facility to assist the BRWA in its awareness raising campaigns to increase subscriptions.

The significant weaknesses in supervision were: (i) failure to focus on social factors, public information and consumer relations until much too late -- no social assessments or consumer surveys were carried out until the end of the project; (ii) lack of candor in Implementation Status Report ratings, which consistently rated performance on development objectives as Moderately Satisfactory or Satisfactory; and (iii) failure to restructure the project following the Mid-Term Review -- which documented the implementation failures and the local institutional and political challenges -- when it became clear that the original objectives and targets were unduly ambitious. Restructuring was left until very late in the project and mainly confined to adding some outcome indicators to focus attention on current priorities. It should also be noted that the shortage of project data in the ICR -- on water quality, maintenance and service standards, the number of unregistered connections -- suggests that more attention was warranted on the part of the Bank team to regular monitoring practices.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The government was unsuccessful in providing the enabling environment for success of the project. The extreme delay until late 2009 in appointing a permanent, professional Director General was a major constraint to developing the capacity of the BRWA. Delay until 2005 in passing the by-laws for merging the utilities into the BRWA was also a constraint but, without a leader for the BRWA, the benefits of the new institutional set up could not be fully realized. Although the government had committed to water tariff reforms at the outset of the project, these were not achieved.

Government Performance Rating: Unsatisfactory

b. Implementing Agency Performance:
The Council for Development and Reconstruction, which held overall responsibility for project management, worked closely with the Bank and the BRWA to resolve issues that blocked project execution, particularly the issues related to procurement and project management that were their direct responsibility. Safeguard and fiduciary procedures were handled adequately, although procurement and contract management capacity was stretched, resulting in some delays and inefficiencies. The performance of the BRWA suffered from the extended vacuum in leadership until late in project implementation.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
Modest
Although the PAD (p. 26) envisaged a rapid social assessment to be carried out at loan effectiveness and a full social assessment prior to mid-term review, the monitoring framework omitted any social or qualitative indicators. In particular, the project objective of a satisfactory water supply was not supported, in the absence of indicators of water quality, the quality of maintenance and service operation, or consumer satisfaction. Population estimates were inadequate to support some of the targets for access.

b. M&E Implementation:
Modest
The planned social assessments were not carried out, as a result of which some important issues and constraints were not revealed until very late in the project (see Section 8b above). The lack of a census on connections (until after closing of the project) also impeded decision making and assessment of results. The ICR reports a focus in monitoring on the environmental requirements for compliance with wastewater treatment standards. Indicators of physical and financial progress appear to have been adequately monitored, although weaknesses remained in the data on unregistered connections. The restructuring in 2009 added some indicators to focus attention on the priority actions needed (increase in the quantity of effluent treated, establishment of volumetric tariffs, and hiring of a managing director for BRWA) but failed to address weaknesses which had become apparent in the overall framework and targets.

a. M&E Utilization:
Negligible. There is no indication from the ICR that M&E outputs were used to inform decisions or policy until the very end of the project, following the 2012 consumer survey.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
This was an environmental category B project. The ICR states that there were no reported divergences or waivers from the Bank's safeguard policies and procedures during implementation. In accordance with requirements in the Environmental Management Plan, the implementing agency developed an implementation plan in 2007 incorporating the following activities: (i) guidance on sludge reuse and the types of crops that could be irrigated with treated effluent; (ii) workshops to be conducted on effluent reuse, monitoring and sampling of water and wastewater quality, along with public awareness campaigns; and (iii) a program to monitor water supply systems, the wastewater treatment plant and treated sludge. These activities were carried out with no reported issues. A total of 106 plots were expropriated to provide land for project works. The ICR team confirmed in 2012 that the landowners were compensated as required by the Ministerial Decree 7848 of April 30, 2002.

b. Fiduciary Compliance:
-- Financial management: According to official records reviewed for the ICR, financial management continued to meet government and Bank standards. All audits were submitted as required and acted on where necessary.
-- Disbursement: There was a temporary blockage in the disbursement of some funds, following early restructuring of the major works contracts. A corresponding change in disbursement categories was requested by the Borrower at the end of 2007 and the Loan Agreement was amended in February 2008 to include a new disbursement category which remedied the problem. No other changes were made in disbursement arrangements and Bank supervision verified that all disbursement requests were properly documented.
-- Procurement: The Bank agreed to revision of the Procurement Plan to merge the water and sanitation infrastructure works into combined contracts. This change resulted in early delays in implementation but resulted in cost savings, allowing the project to cover additional towns and villages in the Ba'albeck area. Bank supervision teams noted weak procurement capacity as a consistent issue throughout project implementation. Measures taken to address this included local training to strengthen procurement capacity and the recruitment early in the project of a project coordinator dedicated to procurement to manage contracts and ensure that Bank guidelines were followed. This contract was not renewed in 2010, following which additional delays and deterioration in the quality of documentation were reported. An Independent Procurement Review in March 2011 concluded that Bank guidelines were generally followed but that weaknesses in the institutional set-up and, more seriously, in contract management resulted in unnecessary delays and missed deadlines.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Unsatisfactory
Unsatisfactory
 
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Borrower Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
-- Modification of consumer behavior is a lengthy process that must be accounted for in the design and implementation of projects (ICR).
-- The implementation of political reforms is a complex process to which project results should be cautiously linked ICR).
-- Supervision teams should not feel trapped by project objectives and outcome indicators that cease to be appropriate in local conditions. Early restructuring could have re-aligned the project to achievable objectives and targets.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

-- The ICR is admirably candid and draws thoughtful and pertinent lessons.
-- Tighter organization of the information into the relevant sections could have provided a clearer picture to the reader and helped to avoid gaps. Some sections are thin: for example, the discussion of monitoring and evaluation reports on findings rather than M&E performance and is restricted to environmental and social assessment; the section on achievement of objectives omits information on relevant outcomes, including information found in other sections of the report (e.g., the economic analysis).
-- There are discrepancies in the figures for project costs, between p.i and p. 27 and between the two tables on p.27.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions