Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Persistent Organic Pollutants (pops) Stockpiles Management & Destruction Gef Project


  
1. Project Data:   
ICR Review Date Posted:
01/31/2013   
Country:
Moldova
PROJ ID:
P090037
Appraisal
Actual
Project Name:
Persistent Organic Pollutants (pops) Stockpiles Management & Destruction Gef Project
Project Costs(US $M)
 12.61  13.41
L/C Number:
Loan/Credit (US $M)
 12.61  13.41
Sector Board:
Environment
Cofinancing (US $M)
 0.00  0.58
Cofinanciers:
CIDA
Board Approval Date
  12/15/2005
 
 
Closing Date
05/31/2010 12/31/2010
Sector(s):
Central government administration (39%), Solid waste management (33%), Petrochemicals and fertilizers (19%), Other industry (9%)
Theme(s):
Pollution management and environmental health (50% - P) Other environment and natural resources management (25% - S) Environmental policies and institutions (25% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Richard C. Worden
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
This is a stand-alone Global Environment Facility (GEF) operation with no direct World Bank Group financing. The Global Environmental Objective (GEO, PAD, page 7) is as follows: “sustainable POPs stockpile management and strengthening of the regulatory and institutional management for long term control of POPs and other toxic substances in line with the requirements of the Stockholm Convention and protocols ratified by Moldova.”

According to the Global Environment Facility (GEF) Trust Fund Grant Agreement (TFGA-055875, Schedule 2, p. 13) the objective is stated as follows: “The objective of the project is to protect the environment and human health by safely managing and disposing of stockpiles of POPs contaminated pesticides and PCBs, and by strengthening the regulatory and institutional management for the long term control of POPs and other toxic substances in line with international conventions and standards.”

This Review will use the objective as stated in the TFGA to assess the project since it is more monitorable.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project was comprised of three components, each with two sub-components and consisting of a number of specified “activities” as follows:

Component 1: Management and Destruction of POPs. (Estimated Cost: US $ 8.23 million; Actual Cost: US $9.33 million). Sub-component 1.1: Destruction of Stockpiles of POPs Containing Contaminated Obsolete Pesticides (total cost: estimated US $5.37 million, GEF: US $1.21 million) to finance the repackaging, transport, and incineration of 1,293 tons of prioritized stockpiles of “obsolete pesticides” (that is, banned pesticides) from 13 warehouses (out of a total of estimated 3,000 tons held in 37 warehouses). Sub-component 1.2: Management of PCBs and Destruction of Obsolete Capacitor Stockpiles (total cost: estimated US $2.86 million, GEF: US $2.58 million) consisted of three activities: (i) Establishing an inventory of PCB containing contaminated equipment; (ii) Destruction of stockpiles of obsolete [electrical] capacitors; and (iii) a Feasibility Study to assess cleaning up the Vulcanesti electricity transmission substation site.

Component 2: Development of a Regulatory Framework for PCB Management and Control. (Estimated Cost: US $2.32 million; Actual Cost: US $2.59 million). Sub-component 2.1: Modification of the Regulatory Framework for POPs Management (total cost: US $0.74 million, GEF: US $0.61 million) was intended to provide the legal basis for POPs chemical management and set up an overall chemical safety system in Moldova according to European Union (EU) regulations and standards. It contained three activities: (i) Assessment of existing legislation regarding compliance with EU regulations; (ii) Assistance in drafting relevant regulations and instructions; and (iii) Developing regulatory mechanisms. Sub-component 2.2: Capacity Building (total cost: US $1.58 million, GEF: US $0.66 million) was intended to strengthen the government’s laboratory capacity through the following three activities: (i) Strengthening inspectors to enforce POP regulations and prevent further accumulation of POP stockpiles; (ii) Upgrading and strengthening existing laboratories to analyze POPs; and (iii) Preventing the accumulation of new stockpiles of obsolete pesticides.

Component 3: Institutional Strengthening and Project Management. (Estimated Cost: US $2.06 million; Actual Cost: US $1.49 million). Sub-component 3.1: Institutional Strengthening (total cost: US $1.43 million, GEF: US $0.74 million) was intended to achieve this purpose by undertaking four activities: (i) design, install and test a POPs Information and Reporting System; (ii) establish a POPs Monitoring Network of national laboratories; (iii) Identify POPs residues and map polluted areas; and (iv) conduct a POPs communications framework to increase public awareness and launch educational outreach activities. Sub-component 3.2: Project Management Support (total cost: US $0.64 million, GEF: US $0.55 million) provided the Project Management Team (PMT) with support to carry out its financial management, procurement, auditing, monitoring and evaluation functions, and to disseminate information about project results.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: The estimated cost of the project was US $12.61 million; actual project cost was US $13.41 million or six percent above the estimated cost at appraisal (ICR, Annex 1: Project Cost and Financing, and Annex 3: Economic and Financial Analysis).

Financing: This was a GEF project (although with US $0.30 million from two related IDA projects in Moldova). Estimated and actual GEF financing was US $6.35 million. This was supplemented by actual parallel financing of US $1.55 million from NATO, US $0.93 million from a Dutch NGO (Milieukintakt Oost-Europe), and US $0.30 million from two related IDA projects in Moldova. Actual financing from these entities was: US $6.35 million from the GEF, US $1.55 million from NATO, US $0.84 million from the Dutch NGO, and US $0.37 from two related IDA projects (Energy II and RISP II). Co-financing from the Canadian International Development Agency (CIDA) for US $581,577 was provided to help remediate areas contaminated with POP pesticides and to clean up PCB contaminated oil used by electrical energy generating companies. All GEF grant monies were disbursed by the end of the project through two minor project restructurings to adjust disbursement categories in the final six months of project implementation (in May and December of 2010 - there are slight discrepancies in the dates given by the ICR).

Borrower Contribution: The Borrower's contribution at appraisal was estimated at US $3.42 million, but the Borrower's actual contribution was US $3.72 million, or nine percent above the estimated contribution.

Dates: The project was approved by the Bank's Board on December 15, 2005 and became effective on March 9, 2006. It was scheduled to close on May 31, 2010 but closed seven months later on December 31, 2010.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial. The project’s objectives are consistent with the World Bank Group’s Fiscal Year 2009-2013 Country Partnership Strategy (CPS) for Moldova, which highlighted the environmental legacy problems of unusually high amounts of PCBs and POPs requiring final disposition under its second Pillar and 7th Strategic Goal of “Improved access to and quality of health, water, and sanitation services and reduced environmental degradation and health hazards” (CPS, pp. 21 and 24-25). The objectives are also aligned with Moldova’s implementation of its Economic Growth and Poverty Reduction Strategy Paper, which had several references to POPs that were contained in stockpiles of obsolete pesticides and PCBs in electrical equipment. They are also consistent with the more recent National Development Strategy (2008 -2011), which seeks to “improve the policy framework in the area of the environment and natural resource use” (ibid). On a global level, the project contributed to Moldova meeting its legal obligations under two other international environmental agreements – the Basel Convention on the Control of Trans-boundary Movements of Hazardous Wastes and Their Disposal, and the Rotterdam Convention on the Prior Informed Consent Procedures for Certain Hazardous Chemicals. Finally, the project contributed significantly to achieving the objectives of the GEF Operational Program (OP14) for reducing and eliminating releases of POPs.

b. Relevance of Design:
Substantial. The causal chain between inputs, outputs, outcomes and achieving the PDO was clear and logical. The two sub-components under the first Project Component (Management and Destruction of POPs) addressed the two distinct sources of POPs contamination in Moldova: stockpiles of obsolete pesticides containing POPs, and equipment (e.g., capacitors, transformers, and oils) used in the electricity generating industry that contained high levels of PCBs. The second Project Component (Development of a Regulatory Framework for PCB Management and Control) and the first sub-component of the third Project Component (Institutional Strengthening) contributed to the achievement of the second part of the PDO to “strengthen the regulatory and institutional management for the long term control of POPs and other toxic substances in line with international conventions and standards” (GEF Trust Fund Grant Agreement (TF-055875, Schedule 2, p. 13)). With one major exception (the amount of contaminated soils at the Vulcanesti electrical substation were estimated at 50 tons, but turned out to be more than 3,000), the estimated amounts of inputs required as well as the expected outputs were accurate and appropriately sized. The Key Indicators in the Results Framework led logically to achieving both of the objectives, as discussed below in Section 4.


4. Achievement of Objectives (Efficacy) :

To protect the environment and human health by safely managing and disposing of stockpiles of POPs-contaminated pesticides and PCBs: Substantial.

Obsolete pesticides containing POPs had been collected previously by Moldova’s Ministry of Defense from more than 350 sites across the country into 37 centralized warehouses. The project financed the adequate re-packaging, transportation, and incineration of nearly 1,300 tons of stockpiled obsolete pesticides from 13 warehouses. This exceeded the target set for this indicator of 1,050 tons with co-financing assistance received from NATO and a Dutch NGO. However, it represents less than 20% of the total estimated amount of obsolete pesticides (6,940 tons) that existed in Moldova prior to the start of the project in 2006, more than half of which (nearly 4,000 tons) are located at a pesticide dump built in 1978 near the village of Cismichioi (PAD, Annex 1, p. 37). Due to funding limitations for this sub-component, the project focused on disposing obsolete pesticide stockpiles at the 10 highest ranked sites in Moldova in terms of the risks they posed to human health and the environment. (The PAD noted on page 12 and in Annex 4 that, “As additional funds become available for disposing of the remaining pesticide stockpiles, it is anticipated that the risk ranking will continue to be used to prioritize their disposal.”)

At the time of project preparation, it was estimated that there were approximately 17,300 obsolete electrical capacitors containing PCBs. These were used to transmit electricity at 13 different substations in Moldova (PAD, p. 31). In addition, two unlined pits of buried capacitors that had exploded in two separate incidents at Vulcanesti substation in the 1970s were discovered. These were estimated to contain another 2,000 capacitors and to have contaminated up to 50 tons of soil beneath the pits. The total PCB content of the 19,200 capacitors was estimated at 355 tons and combined with the weight of the destroyed capacitors was estimated at just over 1,000 tons, the destruction of which would have made “a significant contribution to reducing the total global PCBs problem [in Moldova]” (PAD, p. 31). Work to remove the PCB-contaminated capacitors was completed within a year (Oct. 2006 – Oct. 2007) resulting in the destruction of 937.5 tons from 18,656 PCB contamination capacitors, including 84.5 tons of PCB contaminated capacitors unearthed from the pits found at Vulcanesti substation.

However, the feasibility study for the site clean-up of the Vulcanesti substation revealed that more than 3,000 tons of soils (not 50 as originally estimated) were highly contaminated below the two unlined pits of exploded capacitors (see Section 8 below). This 60-fold increase in the amount of highly contaminated soil forced a re-design of the remediation plan that was eventually implemented. Instead of incinerating the contaminated soil, the project financed the relocation of 350 tons of contaminated soil in two lined cofferdams located at the Vulcanesti site, which were covered with a protective layer of uncontaminated soil and re-vegetated to avoid infiltration of water and minimize the formation of toxic leachate beneath the cofferdams. Then, in the summer of 2010, another 2,725 tons were moved to two additional cofferdams built at Vulcanesti for that purpose. PCBs are highly toxic compounds that are mobile in soils, which they can contaminate along with groundwater aquifers and surface water bodies. While the solution isolated 3,075 tons of contaminated soil, preventing it from posing further risk to human health and the surrounding environment (as long as effective controls are monitored and maintained), it is not a permanent solution to the problem, which will still need to be addressed in the future. Moreover, there is some doubt concerning adequate monitoring since only a "concept design" of the information management and reporting system and monitoring network was completed (see last paragraph of this Section). The objective of managing and destroying the existing stockpiles of PCB contaminated capacitors and other electrical equipment found at electricity transmission substations was substantially achieved.

Strengthening the regulatory and institutional management for the long term control of POPs and other toxic substances in line with international conventions and standards: Modest.

The second specific objective to create a safe and modern chemical safety system in Moldova to manage POPs in full compliance with EU legal standards and directives experienced “significant delays” (ICR, pp. 10) in the preparation and implementation of efforts to modify the regulatory framework to manage POPs. Once draft laws and regulatory standards were proposed, they “underwent significant back-and-forth revisions causing delays in their submission to Cabinet and ultimately, Parliament” (ICR, pp. 10-11). “A total of 15 draft laws and regulations were completed under the project, although none [was] fully enacted” (ICR, p. 16).

There was a greater degree of attainment of the capacity building targets, such as training 280 inspectors to enforce POP regulations and prevent the build-up of future stockpiles, but for several activities these targets provided little useful information in terms of the outcomes achieved. For example, there was no indication of the outcome of upgrading the equipment of existing laboratories in the State Environmental Inspectorate, the Ministry of Health and the Hydro-meteorological Service to detect the presence of POPs in different media (i.e., in air, water and soil samples). Likewise, the project achieved its target of training 110 farmers in Integrated Pest Management practices, but evidence of the impact of this training or of the likelihood of its replication by other farmers was not provided in the ICR.

As part of the project’s institutional strengthening activities, it sought to design a POPs Information System, install the required computer hardware and software, and test the performance of the system. This system was intended to: (i) comprehensively track all the project implementation data; (ii) inventory the packaging, transportation, and destruction of POP-contaminated pesticide stockpiles and PCB-contaminated electrical substation capacitors and equipment in a database; (iii) record the institution of new laws, regulations, and instructional notes; (iv) continually update a digitized website with a GIS map of 1,600 contaminated “hotspots” in the country prioritized on the basis of the risk they posed to human health and the environment; and (v) report on all of this information as part of the project’s monitoring and evaluation system.

However, for reasons that were not adequately explained in the ICR, only “a ‘concept design’ of the POPs Information Management and Reporting System and monitoring network was completed” (ICR, Annex 2, p. 29), which could have implications for the future management and control of toxic substances and the prevention of their stockpiling or loss of containment again in the future. This is a particular cause for concern given that 70% of obsolete pesticide stockpiles and 20% of PCBs remain unmanaged in the country.


5. Efficiency:

An ex-ante incremental cost analysis was conducted at appraisal, showing that the incremental cost of the “GEF alternative” was expected to be an additional US $12.6 million to repackage the POP-contaminated pesticides and consolidate them into 37 warehouses around the country. The ex-post incremental cost analysis showed the actual incremental cost to have been US $13.41 million, or about US $800,000 (6.4%) over the estimated cost for Component 1 activities. Even though more resources were spent on these activities than expected, the target of POP-contaminated pesticides incinerated was exceeded by nearly 150 tons or 10%, and the number of PCB-containing electrical capacitors destroyed was exceeded nearly five-fold (from 6,400 units to more than 30,000). The solution found to deal with the grossly underestimated amounts of highly contaminated soils found beneath two unlined pits at the Vulcanesti substation was efficient in that it was the only feasible manner to handle sixty times the quantity of soil that had been estimated prior to the feasibility study of the site.

More funds were also spent on Component 2 activities than estimated at appraisal (16%) to strengthen the regulatory framework addressing POPs, even though the targets for these activities were only partially achieved. This inefficiency was caused primarily by delays in initiating the work and disagreements between the consulting firm and the Ministry of Environment over the level of Ministry staff support and participation in carrying out the tasks. Conversely, only 75% of the estimated funds available for the third component were spent, particularly to strengthen relevant Moldovan institutions under sub-component 3.1 (actual US $0.58 million vs. estimated US $1.42 million). The POPs Information Management and Reporting System and the POPs Monitoring Network were only designed, not completed as originally proposed in the PAD.

The Cost-Effectiveness Analysis that was presented as part of the Economic and Financial Analysis in Annex 3 of the ICR indicates that “the cost/ton of POPs eliminated was approximately US $4,183/ton. This cost includes repackaging, transport, and incineration (plus inventorying)” (p. 33). It then states that “prices for high temperature incineration of high-halogen solids (e.g., pesticides) using commercially available technologies average US $1,000 – $1,500 per metric ton. If approximately one quarter of total cost is for incineration -- this appears to be cost effective for a nation-wide program." However, there is no indication of international comparisons which could show the 25% of total cost to be a reliable figure.

In summary, time overruns were seven months (13%) and cost overruns US$0.8 million (6.4%). However, the target of POP-contaminated pesticides incinerated was exceeded by 10%, and that of PCB-containing electrical capacitators by nearly five-fold. Only limited legislative aims were attained though more money was spent than anticipated on related activities. CIDA trust funds were mobilized to implement an innovative cofferdam approach to isolate contaminated soils at Vulcanesti and to prioritize 13 of the 37 warehouses with POPs and BPs for incineration to maximize risk reduction of “legacy” environmental problems within constrained resources. On balance, efficiency is rated as substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
%
%
ICR estimate:
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The relevance of both objectives and design was substantial. The project successfully achieved its first goal of safely managing and disposing of stockpiles of POPs-contaminated pesticides and PCBs. Obsolete pesticide stockpiles at the 10 highest ranked sites in Moldova were disposed of, and the removal of PCB-contaminated electrical transmission line capacitors was completed within a year. However, achievement of the second objective -- strengthening the regulatory and institutional management for the long term control of POPs and other toxic substances in line with international conventions and standards -- was modest, as regulatory reform efforts experienced “significant delays,” and the POPs information management and reporting system was only taken to the "concept" stage (which could also have implications for the effective monitoring and control of toxic and contaminated soils in the future). Efficiency was rated as substantial. Overall outcome is assessed as moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The risk that the project’s expected outcomes will be achieved or maintained is assessed as high for the following reasons. First, despite the fact that over 1,150 tons of obsolete pesticides (OPs) were destroyed by the project, there are still nearly 5,800 tons of inadequately managed OPs in the remaining warehouses and pesticide dump near the village of Cismichioi. Second, the cofferdams at Vulcanesti power substation are temporary holding areas intended to “isolate” the contaminated soils from the surrounding environment. However, they must be managed and maintained to ensure their integrity -- a task rendered difficult by the fact that the information and management reporting system had not been taken beyond the "concept" stage by project closure -- and their final disposition or destruction must still be addressed. Third, without adequate, up-to-date laws governing the handling, use and disposal of POPs, there are no legal requirements and restrictions to enforce. Fourth, this is further compounded by the project’s failure to establish a functional information management and reporting system to track these chemicals and wastes systematically, and to guide government actions to remediate existing problems or to prevent future stockpiling of such chemical wastes.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
The Bank helped Moldova prepare a project that was relevant to both the country and the CPS using GEF funds that were used to leverage parallel and co-financing from other donors and NGOs. The project’s design struck an appropriate balance among the three components intended to achieve the project’s objective. With the one exception of the quantity of PCB-contaminated soils in two unlined pits at the Vulcanesti substation, the estimates of obsolete pesticides and PCB-containing electrical capacitors and transformers made at appraisal accurately reflected the actual amounts present in various warehouses and sites across the country. This indicates good preparation and appraisal work on the part of the Bank’s project staff.

Regarding the underestimate of PCB-contaminated soils at Vulcanesti substation, the pre-feasibility study conducted at the preparation stage had to rely on "a first rough estimate of the total amount of PCB in the upper 60 cm of the soil....by simply taking the average of the 14 samples below the assembly and multiply[ing] with the weight of the soil" (PAD, page 33). The PAD noted the "high uncertainty" of the resulting estimate. The fact that the project sponsored a subsequent feasibility study at Vulcanesti suggests that it was recognized that there was inadequate information about the conditions there, motivating an in-depth study which ultimately provided better information about the extent of the problem. Once this had been determined, Bank staff worked swiftly with PMT to find a solution.

The preparation team also noted the importance of starting immediately on making regulatory and legal upgrades to the regulatory framework relating to POPs, which militated some of the delays experienced later during project implementation. Preparation of project documentation was sound and the Results Framework and M&E Plan along with the estimated budget were accurate and realistic.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:
The Bank’s supervision of project implementation was timely, responsive, and appropriate. It interceded in several instances in which its participation was constructive, such as working with the involved parties to resolve misunderstandings and disagreements between the Ministry of the Environment and the consulting firm hired to help them analyze the existing legal and regulatory framework for POPs with the aim of preparing a more adequate and modern chemical management system in Moldova. It was also proactive in implementing a CIDA-funded pilot demonstrating the use of cofferdams to contain PCB-contaminated soils at Vulcanesti when they realized that a very different treatment alternative had to be developed in response to the actual quantities encountered there. The Bank team also restructured the project twice in the final six months of implementation in order fully to disburse all funds and complete as many activities and outputs as possible. All fiduciary obligations by the client and applicable safeguard policies were adequately supervised by the Bank’s in-country TTL and other team members.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government met all of its legal and financial obligations under the terms of the GEF Trust Fund Grant Agreement, and made substantial progress toward meeting its international obligations as a signatory to the Stockholm Convention on POPs. There was good coordination among various Government agencies in preparation of the project and during its implementation, especially between the Ministry of Defense and the Ministry of the Environment. The Government also allowed the NATO “Science for Peace” Project to upgrade laboratory facilities and equipment to detect and analyze POPs, and it allowed a Dutch NGO (Milieukontakt Oost-Europa) to train 110 farmers in Integrated Pest Management practices to reduce the use of pesticides. Counterpart financial contributions were received on-time and exceeded appraisal estimates by US $300,000 or 8.7% (actual US $3.72 million over an estimated US $3.42). The Government also provided substantial in-kind contributions by scaling up PCB dielectric oil sampling in electrical equipment (transformers) to obtain an additional 20,000 samples across the rest of the country, and granted the project a VAT exemption saving it from a 20% tax on goods and services. The only criticism of the Government’s performance was some initial delay in procurement during the first year of the project. Several legal and regulatory reforms were not fully enacted by the project’s closure partly due to frequent government elections and turnover of MOE ministers during the project’s implementation.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
The Ministry of the Environment (MOE) performed well as the Implementing Agency, although there were some misunderstandings and disagreement encountered between MOE and the consulting firm hired to modernize the legal and regulatory POPs framework. These differences were eventually worked out, but there were “significant delays” in initiating the work. In addition to these misunderstandings between MOE and the contractor, “the allocation of insufficient MOE staff to the task” was cited in the ICR (p. 10), and lengthy review processes that slowed down the presentation of draft legislation and regulations to the country’s Cabinet and Parliament. The Project benefitted greatly from the fact that the Project Management Team (PMT) that had managed the previous development of a National Implementation Plan for POPs was retained to manage this GEF-funded project as well. As a result of building upon this experience, all financial management, procurement, and safeguard policies were fully complied with. In addition, due in part to conscientious monitoring of project activities by the PMT, there were no occupational safety accidents or incidents despite the hazardous nature of the chemicals handled, transported, and disposed of by the project.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The four key performance indicators (bullets on pp. 11-12 of the PAD) contained in the Results Framework adequately encompassed the main areas in which the project would track progress toward achieving the project development objective and global environmental objective that were attributable to project activities and outputs. The means by which this data was to be monitored and reported were two-fold. First, the contractor hired to inventory, re-package, transport, and destroy the obsolete pesticides and PCB-contaminated electrical transmission equipment was required to maintain a manifest record-keeping system documenting the status of all chemicals and wastes handled from the moment of identification through to final destruction, which was verified by the PMT to ensure compliance with contractual requirements. The second means by which progress toward Intermediate Outcome and Key Outcome Indicators was tracked was the project’s “POPs Information Management and Reporting System” that was to be set up by the PMT. This management information system was intended to track progress on all aspects of project implementation and serve as an early warning system for corrective actions in situations where progress was lagging behind schedule, and to provide quarterly progress reports and annual meetings to discuss progress made toward outcome indicators.

b. M&E Implementation:
One part of the M&E system performed its function well and as planned in the design of the project: the manifest tracking system included in the contract with the private firm hired to identify and inventory, re-package, transport and destroy obsolete pesticides and PCB-contaminated equipment under Component 1 of the project. Outputs and progress toward indicator targets were measured for project components 2 and 3 by the PMT, even though the POP information system never became operational. The PMT’s efforts largely compensated for the fact that only a “conceptual design” of the POPs information system was developed by the project.

a. M&E Utilization:
Data collected by the contractor hired to carry out the activities under Component 1 provided crucial inputs to implementation decision-making processes, such as the interim solution to the underestimated volume of contaminated soils problem at Vulcanesti. Data collected by the PMT on progress made toward completing required outputs and achieving expected outcomes under project Components 2 and 3 provided feedback to project managers and key stakeholders in a timely manner.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
This was a Category “A” project and Environmental Assessment Operational Policy (OP 4.01) was triggered. No other safeguard policies were triggered. The “A” classification reflected the hazardous nature of the chemicals and wastes that would be handled by project contractors. The ICR reports (page 13) that an Environmental Management Plan (EMP) was developed and included in the conditions of the contract that was signed with the contractors hired to carry out the clean-up operations under Component 1. The EMP included a worker safety program that was independently monitored and verified on a monthly basis by the Project Management Team. A comprehensive safeguards review found that no complaints were filed for any environmental or human health impacts having to do with the handling and destruction of POPs during the project, and compliance with OP 4.01 was rated as “Satisfactory” throughout project implementation (ICR, p.13).

b. Fiduciary Compliance:
Regular financial management reviews indicated that a satisfactory FM system was in place during the project in full compliance with the FM covenants of the Grant Agreement. Internal FM controls and accounting procedures were, according to the ICR, also found to be satisfactory, and annual project audits were unqualified (“clean”) (ICR, p. 12). Procurement processes benefited from previous experience of the Project Management Team with Bank procedures and expectations, and “only minor corrective actions [were] needed by the Bank” (ICR, p. 13). Disbursements lagged behind schedule initially due to delays in launching activities under components 2 and 3, but all funds were fully disbursed by project closing with two minor modifications made to the Grant Agreement in the final six months of the project to complete the remaining unfinished activities.

c. Unintended Impacts (positive or negative):
The alternative temporary solution of placing the unanticipated quantity of PCB-contaminated soils at the Vulcanesti electricity transmission substation into four cofferdams instead of destroying them permanently by incineration means that the risk of an adverse public health or environmental impact still exists and that ensuring the isolation of those contaminated soils must be continually monitored in the future.

d. Other:
None applicable in this project.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
The objective of strengthening the regulatory and institutional management for the long term control of POPs and other toxic substances was modestly achieved, constituting moderate shortcomings. Only a "concept design" of the information management and reporting system and monitoring network was completed (which could also have implications for the effective monitoring and control of toxic and contaminated soils in the future). Furthermore, while a total of 15 draft laws and regulations were completed under the project, none were fully enacted. 
Risk to Development Outcome:
Moderate
High
High risks remain due to the lack of: an operational POPs information system, an approved and enforced legal and regulatory framework, and the final, permanent disposal of PCB-contaminated soils at Vulcanesti. 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Moderately Satisfactory
There were “significant delays” in the execution of legal upgrading activities due to “the allocation of insufficient MOE staff to the task,” and failure to establish an Information Tracking System. See Section 9 for more detailed information. 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:



  • Project staff continuity can build experience that becomes institutionalized within implementing agencies to help them develop new capacities over time and deliver future benefits. By maintaining the original composition of the Project Management Team that had prepared the National Implementation Plan (NIP) for the adequate management of POPs as required by the Stockholm Convention, the project was able to benefit from the expertise and experience developed under that previous work.
  • Having more realistic expectations of what could be achieved in developing new policies, regulations and laws when designing projects was an important lesson learned from this project, especially when those outputs impinge on a project’s ability to achieve other outputs or outcomes. In this case, the project was challenged to provide training to POPs inspectors on best practices to enforce new policies, regulations and laws since they were still not in effect at that time. The presence of good intermediate outcome indicators, such as draft legislation prepared or technical standards developed, which project teams could have had more control over, would have helped alert them to activities or outputs that were lagging behind and preventing other activities and outputs from being achieved in a timely manner. Therefore, it is necessary to have intermediate early warning indicators to signal when an activity or output is not occurring in time to allow other activities which may depend on them to proceed.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The information in the ICR was clearly, succinctly and logically presented. It covered all of the required aspects of project preparation, appraisal and implementation, providing sufficient information for IEG to conduct this assessment. The reasons for the differences in the findings of the pre-appraisal feasibility study concerning the Vulcanesti site and those of the subsequent feasibility study should have been more clearly explained. The ICR should have explained why only a "concept" design of the POPS information management and reporting system was completed.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions