Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Environmental Infrastructure Project


  
1. Project Data:   
ICR Review Date Posted:
01/13/2014   
Country:
Moldova
PROJ ID:
P074139
Appraisal
Actual
Project Name:
Environmental Infrastructure Project
Project Costs(US $M)
 9.9  0.77
L/C Number:
Loan/Credit (US $M)
 7.76  0.77
Sector Board:
Environment
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  05/29/2007
 
 
Closing Date
12/15/2011 12/30/2010
Sector(s):
Sewerage (96%), Central government administration (4%)
Theme(s):
Pollution management and environmental health (50% - P) Water resource management (25% - S) Environmental policies and institutions (25% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Stephen Hutton
Ridley Nelson Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The objectives of the project as stated in the Global Environment Facility Trust Fund Grant Agreement (page 5) were: "to improve the quality of sanitary services in Soroca and reduce the discharge of pollutants, including nutrients, from Soroca municipal sources that flow into the Nistru River and demonstrate and disseminate cost-effective nutrient reduction strategies and technologies for municipal wastewater sources."

The project development and global environmental objectives of the project stated in the Project Appraisal Document (page 10) were to "(i) improve the quality of sanitation services in Soroca; (ii) reduce the discharge of pollutants, including nutrients, from Soroca municipal sources that flow into the Nistru River and, subsequently, into the Black Sea; and (iii) demonstrate and disseminate through feasibility studies and workshops, cost-effective and affordable technologies for municipal wastewater treatment for the potential benefit of similar projects for Moldova's existing wastewater treatment plants, for those towns in Moldova that have no wastewater treatment, and for the countries that drain into the Black Sea.

Following IEG practice, this project evaluates against the objectives as stated in the Trust Fund Grant Agreement.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1-A: Wastewater management in Soroca (appraisal estimate $7.89 million, actual $0.77 million).
This would finance rehabilitation of the wastewater collection system and sewerage pipelines, construction of a wastewater treatment facility using constructed wetlands, and 6 months of operation of the facility.


1-B: Engineering consultant and Technical assistance (appraisal estimate $1.48 million, actual $0).
This would provide technical assistance sewer network design, procurement, supervision, and operational assistance, and feasibility studies and pre-feasibility studies for replication in additional towns.

2: Dissemination and replication component related to constructed wetlands (appraisal estimate $0.1 million, actual $0).
This would finance seminars and workshops in which to disseminate knowledge and experience gained from creation and operation of the new facilities.


3: Institutional strengthening component (appraisal estimate $0.15 million, actual $0).
This would finance development of a communication strategy and capacity building for a media campaign to prepare for increasing sewage treatment charges, and training for operational efficiency improvements.


4: Project management (appraisal estimate $0.28 million, actual )
This would support management and implementation by the project implementing unit.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
At appraisal, the total project cost was estimated at $9.9 million. Actual expenditures were only $0.077 million, on engineering design work. No other activities were implemented.


The project was to be financed by $4.56 million grant from the Global Environment Facility and $3.21 million of IDA funds from the Pilot Water Supply and Sanitation Project. Actual disbursements from the GEF grant were $0.77 million, and no IDA funds were disbursed..


The appraisal document notes (page 14) that the borrower was expected to contribute $1.3 million over FY08-11 and that the Soroca Municipality would provide the land for the treatment plant free of charge (valued at $0.77 million). No actual borrower contributions were made.

The project was originally scheduled to close on December 15, 2011. Due to a lack of implementation progress, the project was closed early and the grant canceled on December 30 2010.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Nutrient pollution in the Nistru River was a significant problem for Moldova. The river is the main source of drinking water for Moldova and part of Ukraine, but faces pollution from municipal, industrial, and agricultural sources. Downstream communities on the Nistru had a combined population of 1.4 million. Heavy nutrient loads from these sources contribute to severe eutrophication of the Black Sea, harming fishing and tourism. About 2% of the nutrient loads to the Black Sea came from Moldova.

The quality and reliability of water and sanitation services in Moldova have deteriorated due to aging and under-maintained infrastructure. Service access was particularly low in small-medium sized towns, with 60 percent of residents having access to centralized water supply and 35 percent to sewerage service. Wastewater infrastructure had insufficient capacity and did not meet effluent standards; only 4 of 100 plants were able to meet discharge standards. The institutions responsible for water and wastewater services were weak and financially unviable, with insufficient revenue to cover maintenance.

Soroca was one of 5 cities benefiting from an ongoing Bank financed Pilot Water Supply and Sanitation Project, and so was a good potential candidate for piloting and demonstrating wastewater improvements. The sewer system was 40 years old and had received very little maintenance, and several sections were near collapse, so improving sanitary services was a priority.

The objectives were relevant to the 2004 Country Assistance Strategy operative at approval, which contained a pillar on Improving Access to Social Service, Capital and Community Infrastructure, and Minimizing Environmental Risks, had specific mention of focusing on addressing deterioration in critical infrastructure, and identified provision of adequate water and sanitation services as one of the greatest challenges for Moldova, with a focus on medium sized towns and villages. The objectives were remained relevant to the FY09-12 Country Assistance Strategy operational at closure, which specified that ongoing support for improved water and sanitation and achievement of the corresponding MDG would continue to be a priority.

Relevance of objectives are rated High

b. Relevance of Design:
The project design was likely to achieve the project objectives if implemented satisfactorily. The project was envisioned as a pilot that could then be replicated in other parts of Moldova or nearby countries. It would support direct rehabilitation of the sewerage system in Socora, including construction of a constructed wetland to treat wastewater, and technical assistance of the water institution to improve the capacity to operate and maintain the facilities. This would directly improve quality of sanitary services in Soroca and reduced discharge of pollutants. The constructed wetland technology has been widely used in other countries and is generally effective. The project would provide some support for communication and media work related to an increase of a sewage treatment surcharge needed to provide revenue to the water institution. And it would support dissemination of nutrient reduction through workshops and seminars, which could help encourage replication for other cities.

Relevance of design is rated Substantial


4. Achievement of Objectives (Efficacy) :

During implementation, disputes on land registration and opposition to the use of the land for the waste water treatment plant re-emerged, contributing to a lack of implementation progress and subsequent cancelation of the grant.

Objective 1: Improve the quality of sanitary services in Soroca: Negligible

The only project output was engineering work on design of the wastewater treatment plant; the plant was not constructed and so no improvement in sanitary services was made.

Objective 2: Reduce the discharge of pollutants, including nutrients, from Soroca municipal sources that flow into the Nistru River: Negligible

The project had no impact on the discharge of pollutants.

Objective 3: Demonstrate and disseminate cost-effective nutrient reduction strategies and technologies for municipal wastewater sources: Negligible

No demonstration or dissemination activities were conducted

5. Efficiency:

The project was cancelled without producing any significant outputs or benefits, and so the effort of preparing the project was largely wasted.

Efficiency is rated Negligible.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Though the objectives of the project were relevant and consistent with country strategies, and the design if implemented was relevant and so likely to achieve the project outcomes, implementation issues meant that almost no outputs were produced, and so the efficacy and efficiency of the project were negligible. Together these lead to an outcome rating of Highly Unsatisfactory.

a. Outcome Rating: Highly Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

The operation was canceled without significant activity, and so the risk to development outcome is not evaluable.

a. Risk to Development Outcome Rating: Non-evaluable

8. Assessment of Bank Performance:

a. Quality at entry:
The ICR notes (page 14) that Bank inputs and processes during preparation were adequate: the preparation team included sufficient expertise; project preparation was not rushed; technical, economic, and financial analyses were conducted; the borrower's fiduciary capacity was assessed, and a capable project implementation unit was assigned. The appraisal document (page 12) claims that client ownership was created at all levels of government by preparing the project in close collaboration with local stakeholders. The Ministry's construction agency took a lead in project preparation and was involved in the decision making process for the choice of treatment technology, as was the water country. Public consultations were conducted by Bank Environmental and Social specialists. However, there was no strong local champion for the project.

An official QAG quality at entry assessment conducted in November 2007 rated the project as satisfactory, praising the design for appropriate choice of treatment technology (constructed wetlands) that would be affordable for the local community, the inclusion of a replication strategy from the outset, and the use of a project implementation unit from a previous water sector project. The technology choice was based on a least cost analysis, compared to three alternatives. According to the analysis, constructing a treatment plant using the more advanced Activated Sludge system would have required raising water tariffs by an estimated 170 percent.

A QAG Quality Assessment of Lending Portfolio assessment in 2010 rated the quality of project design as moderately unsatisfactory, based on weaknesses in M&E arrangements and in the financial and economic analysis.

At appraisal, it was believed that sufficient land would be available for the wetlands. The Egoreni local council that originally owned the land for the site had voted before project approval to unconditionally transfer the land to the Soroca municipal government. The Bank stipulated in negotiations that the Government would provide cadastral documents that registered ownership of the land by Soroca and evidence of the transfer from Egoreni, and this was done, though it turned out later that this documentation was not sufficient.

Some important risks to implementation were considered and addressed in the PAD (such as on risks from higher water tariffs or inadequate maintenance experience), but several key risks were not identified or addressed or were underestimated. The risk of stakeholder opposition to the proposed constructed wetlands was not identified. The risk of objections to the choice of technology was not assessed, though it had been a subject of much contention during preparation. Risks relating to land transfer and registration were not highlighted during preparation, but completion of land transfer was made a condition of effectiveness.

The ICR notes (page 6) that social analysis focused on affordability aspects and did not adequately address issues of acceptability of the new low-cost technology in the Moldova context. The ICR argues (page 15) that the Communication Strategy should have been implemented in parallel with the technical design preparation and the broader stakeholder consultation.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
The Bank undertook regular supervision missions, and these were adequately staffed. The ICR notes that supervision reports were well written and highlighted key issues.


The Bank undertook some actions to try to resolve implementation delays. The Bank team worked with local stakeholders to discuss the merits of the created wasteland technology for the plant, and organized demonstrate study tours to similar plants in Italy and France, though stakeholders from the locally affected village were not included in the tour. The second of these tours was successful in generating reaffirmed Government support for the project. As they realized it would be a problem, the Bank team made completion of land transfer a condition of effectiveness and later disbursement. Failure to meet this condition triggered cancellation of the grant.

However, the Bank was slow to act on the project's critical problems, choosing to defer decisions in the hope that a new government would make changes; disbursements were only suspended in October 2010, 3 years into implementation, though the Bank team had recommended considering cancelation as early as January 2008.

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The local and central government were supportive to the project during preparation, and there was close coordination between the relevant central and local government officials and the Bank, though government could have done more to ensure timely resolution of the land transfer and registration issues prior to effectiveness.


A period of political uncertainty following indecisive elections in April 2009 contributed to delays, as did concerns with the technology choice for the wastewater plant raised by a newly appointed government minister. The ICR notes (page 16) that after the elections, some individuals were allegedly politically motivated to oppose the project, and encouraged opposition to it at the village level. Though agreements were made between the national government and the Bank on a way forward leading to temporary resumption of implementation, the government was unable to resolve land usage issues and consequently the project was cancelled.

Government Performance Rating: Unsatisfactory

b. Implementing Agency Performance:
The project implementation unit carried out its roles during preparation and implementation, including financial and project management tasks. The implementation unit could have been more proactive in facilitating consultations with stakeholders to resolve issues on technology choice and land issues.

During implementation, the unit was transferred between agencies, which caused implementation problems as the director of the new agency had reservations with the project design and was not supportive of the project. At one point the director of the implementation unit was dismissed without consultation with the Bank. The project stalled in mid 2008 when the agency director terminated the contract for design of the treatment plant, and subsequently failed to provide an action plan requested by the Bank. After the unit was transferred a second time to a new agency, an action plan was submitted, but land issues could not be resolved.

Implementing Agency Performance Rating: Unsatisfactory

Overall Borrower Performance Rating: Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The results framework was extremely simple, containing only three indicators: the amount of wastewater treated, the number of cases where project outcomes were replicated, and the disbursed amount of the wastewater treatment construction contract. Targets were not specified for the first two indicators. Baseline values of zero were implied for all indicators. Weaknesses in M&E were noted by QAG in both a quality at entry assessment and a quality of lending portfolio assessment.

This simple design was not enough to fully assess achievement of the objectives. The indicators were insufficient to assess the achievement of objectives; they measured only the quantity of treated wastewater, and not the actual discharge of nutrients or other pollutants. They did not assess whether members of Soroca actually received improved sanitary services directly.

M&E was to be conducted in conjunction with that for the Pilot Water Supply and Sanitation Project, which shared a project implementation unit.

b. M&E Implementation:
M&E was very limited as no significant investments were carried out.

a. M&E Utilization:
There was no utilization of M&E.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The project triggered safeguard policies on Environment (the project was assigned category B) and on International Waterways. Risk assessments and mitigation measures were carried out as part of the environmental impact assessment process. Issues concerning the location of the wastewater treatment facility and the choice of technology were discussed as part of the process. Due diligence on land ownership was conducted during appraisal, and resolution of land ownership issues was a condition of project effectiveness. Documents confirming resolution of the issue were provided by the government prior to effectiveness, but local opposition and
political disputes concerning the use of land emerged during implementation contributing to cancelation of the project.


The project was assessed as being excepted from the international waterways notification requirement, as the impacts on the waterway were judged to be solely positive (through reduced pollution).

No significant activities were carried out and so the project had no impact on the environment or on waterways.

b. Fiduciary Compliance:
Only one contract was procured under the project, for detailed design of the wastewater plants. Procurement was carried out in accordance with Bank guidelines.

The project implementation unit was responsible for financial management of the project. Financial management arrangements were deemed to be satisfactory in the Bank's supervision reports. A waiver was granted for producing audited financial statements for FY 2010 as no investments were carried out.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Highly Unsatisfactory
Highly Unsatisfactory
 
Risk to Development Outcome:
Non-evaluable
Non-evaluable
 
Bank Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Borrower Performance:
Moderately Unsatisfactory
Unsatisfactory
IEG concurs with the analysis in the main body of the ICR, which assigns a Borrower performance rating of Unsatisfactory based on the Government's inability to resolve the land dispute, which led to cancelation of the project. 
Quality of ICR:
 
Exemplary
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR identifies three key lessons from the experience:
  • Risks related to political economy of projects should be identified and realistically addressed during project preparation. In this project, strong political opposition led to cancelation of the grant. If the existence of potential opposition is not identified during preparation, then it cannot be mitigated and may lead to surprises during implementation.
  • When it is recognized that critical implementation issues cannot be resolved, the Bank should be proactive in deciding to withdraw from the project or to adopt other alternative measures. In this project, the Bank chose to defer action, which led to an extended period of inaction before the project was canceled.
  • Low-cost technologies may face opposition if they are perceived to be inferior, even when technical and economic analysis supports their use. In this project, constructed wasteland technology was new to Moldova, and many people preferred higher quality but more expensive options that had been in use in the Soviet era. The Bank faces reputational risk if it is perceived as overselling specific technologies that are resisted by stakeholders.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is very well written and clear, and provides a high quality critical analysis of project preparation and quality at entry issues, and a thorough description of the implementation issues that led to cancellation of the grant. Though the project was unsuccessful, the ICR distilled useful lessons from the experience.
The only issues with the ICR are very minor. There is a discrepancy in ratings of Borrower Performance in the ICR; the text (page 17-18) cites Performance as Unsatisfactory, while the front matter assigns a rating of Moderately Unsatisfactory. The ICR refers to quality of portfolio assessments as being conducted by IEG, but in fact these were conducted by QAG. There is a discrepancy between actual project cost (Annex table 1a $0.076 million) and the actual project financing (Annex table 1b and footnote 6 $0.77 million). The ICR did not discuss the Relevance of Design under the heading Relevance of Objectives, Design and Implementation.

a. Quality of ICR Rating: Exemplary

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions