Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Carib-gef-implementation Of Adaptation Measures In Coastal Zones


  
1. Project Data:   
ICR Review Date Posted:
11/08/2013   
Country:
Caribbean
PROJ ID:
P090731
Appraisal
Actual
Project Name:
Carib-gef-implementation Of Adaptation Measures In Coastal Zones
Project Costs(US $M)
 5.44  4.30
L/C Number:
Loan/Credit (US $M)
 2.10  2.03
Sector Board:
Environment
Cofinancing (US $M)
 1.84  1.35
Cofinanciers:
Caribbean Community Climate Change Center St Lucia Coconut Bay Resort, GTZ, St Vincent Fish Complex, Meteorological Research Institute of Japan, Government of Italy.
Board Approval Date
  09/07/2006
 
 
Closing Date
06/30/2011 12/31/2011
Sector(s):
Central government administration (58%), General agriculture fishing and forestry sector (14%), General water sanitation and flood protection sector (12%), Flood protection (8%), Water supply (8%)
Theme(s):
Climate change (29% - P) Biodiversity (29% - P) Water resource management (14% - S) Land administration and management (14% - S) Environmental policies and institutions (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
J. W. Van Holst Pellekaan
Roy Gilbert Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The Project Appraisal Document (PAD) for the "Implementation of Adaptation Measures in Coastal Zones Project" (part of the UN Special Adaption to Climate Change Project or SPACC) states that the Project Development Objective (PDO) was "to support efforts by Dominica, St. Lucia, and St. Vincent and the Grenadines to implement specific (integrated) pilot adaptation measures addressing primarily, the impacts of climate change on their natural resource base, focused on biodiversity and land degradation along coastal and near-coastal areas" (para 21).

The PAD also states that "the project seeks to produce knowledge of global value on how to implement adaptation measures in small island states that can be applied in other countries in the region, not participating in the project and even for islands in other regions of the world" (PAD, para 22). This is the Global Environment Objective (GEO).

The Global Environment Facility (GEF) Grant Agreement states that the objective of the project is "to assist the Recipient in supporting the efforts of the Participating Countries to implement specific, integrated, pilot adaptation measures that address primarily the impacts of climate change on the Participating Countries’ natural resource base, with a focus on biodiversity and land degradation along coastal and near-coastal areas" (Schedule 1 on page 7). The Grant Agreement is silent on the GEO.

The project objectives in the PAD and the Grant Agreement are materially the same. An important difference, however, is presentation. The PAD identifies the three Participating Countries in the statement of objectives whereas the Grant Agreement defines them in the Appendix to the Agreement (page 18).

For the purpose of this Review the PAD's statement of the PDO will be regarded as the project's objective because its presentation is more transparent.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The PAD states that the project supported three activities prioritized in national adaptation strategies and refined through a series of regional and national consultations (para 25). These, together with project management, make up the project’s four components described in detail in paras 26 to 30.

Component One: Identification, evaluation, selection and design of priority adaptation measures addressing impacts of climate change on biodiversity and land degradation (estimated total cost US$1.49 million of which GEF funding US$0.20 million; actual cost US$1.23 million of which GEF funding US$0.37 million). Under this component the following activities were supported:

Sub-component 1 - Feasibility analysis of proposed adaptation measures. Supported (a) assessment of the current ecosystem and physical condition and trends in coastal areas of the Participating Countries (PCs); (b) provision of data by the Meteorological Research Institute of Japan (MRI) that estimated current and future temperature and rainfall patterns along proposed project sites; (c) analysis of specific climate change impacts, primarily concerning biodiversity of global significance and assessment of alternatives including, technical, environmental, social, institutional, and risk and cost analyses; and (d) selection of recommended actions.
Sub-component 2 - Design of adaptation measures. Supported (a) technical, engineering design; (b) environmental and social management plans; (c) community participation; (d) M&E system design to measure the impacts of adopted measures; (e) establishment and adoption of plans and cooperation agreements by PCs with local and regional institutions and agencies for implementation of adaptation measures.

Component Two: Implementation of selected adaptation measures was designed to address climate impacts on biodiversity and land degradation (estimated total cost US$2.35 million of which GEF funding US$1.5 million: actual total cost US$2.18 million of which GEF funding US$1.36 million). The project was to support the coordination and implementation, of pilot adaptation measures at seven sites.

Sub-component 1 - Coordination of adaptation measures in the participating countries. Supported the coordination and implementation of selected adaptation measures in seven sites namely (a) provision of data for analyzing the pressures on biodiversity and land degradation from anticipated climate projections; (b) in-country coordination and supervision by PCs of implementation of adaptation measures in consultation with communities.
Sub-component 2 - Implementation of adaptation measures in the biodiverse Morne Diablotin National Park and its Neighboring Communities (Dominica) identified among updated management plans under Component 1. Supported adoption of a Park Management Plan to address the impacts of global climatic change (GCC) primarily on globally important biodiversity and on land degradation issues and to reduce vulnerability in the Morne Diablotin National Park and in its neighboring coastal communities of Colihaut, Dublanc and Bioche. Activities were to maintain ecosystem integrity and prevent biodiversity losses.
Sub-component 3 - Morne Trois Pitons National Park Integrated Ecosystem Management (Dominica): Supported adaptation measures to address the impacts of GCC primarily on globally important biodiversity and on land degradation issues and to reduce vulnerability in the Morne Trois Pitons Nation Park, a World Heritage Site. Activities were to maintain ecosystem integrity and prevent ecosystem fragmentation.
Sub-component 4 - Sustainability of Water Resources and Supply of the Vieux-Fort Region (Saint Lucia): Complemented the national water supply program by adding a climate overlay and establishing adaptation measures to increase resilience to the impacts of climate variability and climate change in the Vieux-Fort Region of Saint Lucia, and the proposed Pointe Sable National Park.
Sub-component 5 - Strengthened critical coastal infrastructure in the Castries area (Saint Lucia). Pilot adaptation measure to demonstrate the design and implementation of interventions to reinforce critical coastal infrastructure (hospitals, shelters, fire stations and storm barriers) against the effects of intense hurricanes, in the Castries region of Saint Lucia. The project was to provide the scientific and engineering services required to assess vulnerability, define priorities and retrofit one specific building.
Sub-component 6 - Integrated Ecosystem Approach to Climate Change in Bequia and Union Islands (St. Vincent and the Grenadines): Support national efforts to integrate climate change concerns into ecosystem management. It focused on adaptation measures to address the absence of fresh water (Bequia) and coastal vulnerability (Bequia and Union).. This component was also intended to support the completion of Island Integrated Resource Management Plans and implementation of key adaptation measures to address water supply issues and vulnerability to climate change of remaining Black Mangrove stands (Avicennia sp.) in Union Island.
Sub-component 7 - Climate Change Risk Management for Spring Village (St. Vincent and the Grenadines): Support implementation of adaptation measures to address land degradation and its impact on sensitive coastal and marine ecosystems, including coral reefs affected by increased sea surface temperatures and sea level rises, based on the coral reef monitoring program supported through MACC through land use/land planning.

Component Three: Strengthen National Capacity to Implement Multiple Multilateral Environmental Agreements (MEA) (estimated total cost US$0.78 million of which GEF US$0.2 million; actual total cost US$0.08 million of which GEF US$0.08 million). The component, financed by third party and counterpart contributions, goods and services required to develop and establish the building blocks to address many convention objectives as adopted by the three governments and promoted by the UN Framework Convention on Climate Change (UNFCCC), Convention on Biological Diversity (CBD) and UN Convention to combat desertification and drought (UNCCD).

Sub-component 1- Development of a harmonized national reporting framework (i.e. legal, institutional and management structures) by each PC that integrates climate change, biodiversity and land degradation obligations under the UNFCC, the CBD, and the UNCCD coordinated among the three PCs.
Sub-component 2 - Provision of technical assistance by IUCN to Participating Countries to develop a single report using the harmonized national reporting framework.
Sub-component 3 - Testing of the application of the harmonized framework in relation to specific adaptation measures under Part 2 of this Project.

Component Four: Project Management (estimated total cost US$ 0.82 million of which GEF funding US$0.20 million; actual cost US$0.81 of which GEF $0.22 million). This component was intended to support the following:

Sub-component I - Overall technical coordination of Project Activities (incl. implementation of a technical monitoring system), the administrative and financial management of the Project, and annual audits.
Sub-component 2 - Operational expenditures directly related to the implementation, management and monitoring of the Project which expenditure would not have been incurred without the project.

Restructuring

Following the mid term review in September 2010 the project was restructured on October 14, 2010 by deleting the following two sub-components:

Component 2, Sub-component 6 - Integrated Ecosystem Approach to Climate Change in Bequia and Union Islands (St. Vincent and the Grenadines): The part intended "to support the completion of Island Integrated Resource Management Plans and the implementation of key adaptation measures to address water supply (Bequia) and reduce vulnerability to climate change impacts on remaining Black Mangrove stands (Avicennia sp.) in Union Island" was deleted. The reason was that "land tenure, not identified during previous preparation efforts, became a serious obstacle that prevented any effective action" (Restructuring Paper, para 10).

Component 2, Sub-component 7 - Climate Change Risk Management for Spring Village (St. Vincent and the Grenadines): All of this sub-component was deleted because the pilot "faced a complexity of associated land use planning and management issues" (Restructuring Paper, para 10).

Restructuring also resulted in abandoning all but one of the original PDO/GEO indicators and replacing them by indicators which would “better reflect measurable results for which the project could be held accountable” (ICR, para 11).

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs: Total costs were reduced from the original estimate of US$5.44 million to US$4.30 million due to reductions in financial contributions by cofinanciers as described in the next section.

Financing: Annex 1 of the ICR did not provide the standard financing table but this information was subsequently provided by the task team leader. It showed that project financing changed after Board approval as follows (i) a reduction in the contribution of the MRI of Japan from an estimated US$1.39 million to an actual US$0.6 million; (ii) the global financial crisis led the IUCN to cancel its contribution of US$125,000 for harmonization of national reporting frameworks (Component 3.2); and (iii) the effects of the global financial crisis also led to a reduced cofinancing by the three governments from US$1,500,000 to US$850,000 (ICR, page 33).

Borrower Contributions: The three PCs pledged a contribution of $1.5 million to the cost of the project. However, they also experienced budget pressures as a result of the GFC and reduced their total contributions to $0.85 million. This reduction was to some extent compensated by "in-kind" contributions from the Governments but the ICR provides no information on the amount (ICR, para 39)

Dates: The restructuring in October, 2010 was associated with an extension of the closing date from June 30, 2011 to December 31, 2011.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High
The objective of this project is relevant because climate variability and intensification of hurricanes pose a significant threat to the sustainable development of small island developing states. There has been an increase in the intensity and number of hurricanes in the Caribbean Basin since 1970, associated with warmer sea surface temperatures (PAD, paragraph 7). Dominica, Saint Lucia, and St. Vincent and the Grenadines have all been subject to the effects of climate change. The PDO was also relevant to the Regional Partnership Strategy (RPS) for the Organization of Eastern Caribbean States (OECS) with the World Bank for FY06 to 09 in which the second pillar was "reducing vulnerability by ...strengthening risk disaster management" (para 15). The project's PDO was therefore highly relevant to development strategies of Caribbean governments and to the World Bank’s assistance strategy. The PDO remained relevant to Caribbean governments' and Bank strategies in the 2008 RPS Update, the latest RPS dated May 3, 2010 and its Progress Report dated April 1, 2012.

b. Relevance of Design:
Original Design - Modest.
The project’s design included relevant activities and components. However, the project was not well designed to achieve policy and regulatory change related to climate change and the protection of coastal areas. For example, in the installation of a renewable wind power generation facility to reduce operation costs for a desalinization plant assumed that excess power could be sold to the national power company but there was no provision to ensure that the company's policies would be changed to allow it to purchase excess power. The quality of the original results framework was weak because it did not provide a clear chain of actions to achieve the project’s intermediate outcomes or final objectives. The pilots were expected to address complex issues such as land use plans for buffer areas, measurement of water supply deficits, and analysis of flood risks. For a number of pilots the project itself was required to first prepare feasibility assessments (see PAD, para 62 and Annex 4). The project was organized such that the responsibility for feasibility studies and detailed pilot project designs was delegated to country institutions. However, the designated institutions had inadequate capacity to undertake these tasks. Their weak capacity led to considerable delays. Another design issue was that the project sought to blend national and regional perspectives of adaptation to climate change - which were not always compatible

Revised Design - Modest
The restructuring paper recognized the shortcomings of the original results framework (Restructuring Paper, Annex 4) and revised a number of project indicators. The revision of indicators lowered the threshold for the achievement of the project’s objective and simplified them. While indicators became more explicit and simple, many of the revised indicators focused on outputs rather than outcomes (see Restructuring Paper, Annex 4 and ICR, Table 1). Nevertheless, the basic problem of weak design, namely the formulation and implementation of the pilots by the typically weak institutions in the participating countries remained.


4. Achievement of Objectives (Efficacy) :

The project's objective was to be achieved "through (i) the detailed design of pilot adaptation measures to reduce expected negative impacts of climate change on marine and terrestrial biodiversity and land degradation; and (ii) the implementation of pilot adaptation investments" (PAD, page iv). This Review considers the extent to which the project's objectives were achieved and whether the achievements were attributable to the project..

Dominica

(a) Outputs

Ecological Data to Support Adaptation Measures. A comprehensive database was prepared by the Ministry of Agriculture and delivered to the Agriculture Information, Communication and Technology Unit. This output was achieved.

Meteorological Data to Support Adaptation Measures. The two meteorological stations were purchased and installed, as part of the ongoing Caribbean Hydrological Cycle Observing System Project covering the Island States, and information was being collected. This output was achieved.

Investment in Irrigation to Complement Adaptation Measures. The Ministry of Agriculture developed a pioneer irrigation pilot in the Milton area to encourage farmers to diversify away from their encroachment on parkland. It also piloted water users associations for the first time in the Caribbean. A report on results was not prepared before the project closed. Without more evidence it cannot be assumed that this pilot's objective was achieved before it closed.

(b) Outcome

Plans for Adaptation Measures to Climate Change in Two National Parks. The Park Management Plan for the Morne Diablotin and Morne Trois Pitons National Parks were completed and extensively reviewed. These measures comprised a land use plan for buffer areas between areas used for commercial farming and the natural ecosystem, one new meteorological station in each of the two parks, and a comprehensive data base on key ecological variables to measure changes in biodiversity, The plans included all these adaptation measures. It is too early to measure the impact of the plans on natural resource management and biodiversity. However, completion of the Park Management Plans met the project’s objective of implementing specific adaptation measure to address the impact of climate change on natural resources. This achievement was attributable to the project and rated substantial.

Saint Lucia

(a) Outputs

Legislation to Enforce Water Harvesting. The legal framework to enforce rainwater harvesting on new tourist developments in St. Lucia was fully prepared. When the project closed its enforcement was awaiting a thorough review by the Ministry of Physical Planning and Environment before submission to Cabinet.

(b) Outcomes

Investment in Rain Water Harvesting as an Adaptation Measure. The rain harvesting and recycling system at the Vieux Fort Hotel was successfully completed. Tanks with a capacity of 45 cubic meters are used daily by the hotel, reducing the consumption from the water utility. This result has reduced the pressure on St Lucia’s water resources and is an important pilot adaptation measure to reduce the negative effects of climate change on lowering rainfall and or reducing the availability of underground water supplies. This outcome was attributable to the project and its efficacy was rated substantial.

Investment in Recycling Waste Water as an Adaptation Measure. All waste water produced by the Vieux Fort hotel is treated and subsequently re-used for garden irrigation, with no outflow to the coast. This investment successfully addressed the pilot project’s objective of reducing the negative impacts of land degradation along coastal areas on St Lucia due to the discharge of waste water from the hotel. It was an important adaptation measure to the degradation of coastal areas by waste water. This outcome was attributable to the project and its efficacy was rated substantial.

Infrastructure as an Investment in Adaptation Measures. The Ministry of Physical Planning and Environment has disseminated the experiences gained with the Marchand community building pilot which withstood Hurricane Tomas. This provided important information for the design and construction of other such buildings as adaption measures in Saint Lucia where people can take shelter from hurricanes resulting from climate change. This outcome was an important contribution to the objective of implementing pilot adaptation investments and was attributable to the project. Its efficacy was rated substantial.

St. Vincent and Grenadines

(a) Output

Investment in Adaptation to Water Supply Stresses. The Minister of Health was the main counterpart for the preparation of a report on a scheme to manage the adaptation of domestic water supply to stresses related to climate change. The scheme has not yet been fully assembled and a final workshop and technical note have not been completed. A final result was anticipated at or shortly after the project closed.

(b) Outcomes

Investment in Desalinization as an Adaptation Measure. The desalinization plant at the Bequia Fisheries Complex was fully installed and has produced potable-quality water since July 2011 using solar energy. This pilot demonstrated the feasibility of investing in the desalinization of groundwater for commercial purposes as an adaptation to climate change. This was an important outcome attributable to the project. Its efficacy was rated substantial.

Cost Recovery for Sustaining Adaptation Measures. The sustained financial viability of the Bequia desalinization pilot will only be assured through the collection of user fees for the water but at the project's close the purchase agreement of the plant by the St Vincent National Electric Company (VINLEC) had not yet been concluded. This pilot adaptation investment contributed to the project’s objective of adapting to climate change but, given the incomplete status of cost recovery arrangements, the sustainability of the outcome was not yet achieved. The efficacy of this pilot's outcome was therefore rated as modest.

5. Efficiency:

The PAD contains "an incremental cost analysis" intended to show that the expected qualitative additional benefits from the project will exceed the additional costs (Annex 11). The ICR’s annex on economic analysis uses the same methodology as the PAD but also uses the same total incremental costs at appraisal, despite changes in the project's sub-components. The annex provides no useful additional information for assessing the net benefits from the project.

The PAD stated that "A preliminary economic analysis of selected adaptation measures in each component will be undertaken to illustrate the economic efficiency of the type of adaptation measures covered by the project. The analysis will assess direct economic benefits and costs associated with the adaptation measures applying standard economic methodology" (PAD, Para 60).

The ICR acknowledged this plan for economic analysis of pilots but provided no information on whether they were done. However, following discussions associated with this Review the TTL provided the analyses for four of the six pilots which showed they were cost-effective. For the other two pilots (a comparison of the costs of photovoltaic vs wind power sources for the Bequia desalinization plant and an economic analysis of the irrigation pilot) it was concluded that the Bequia plant used the most cost-effective source of power but an analysis of the irrigation pilot was not available.

The restructuring eliminated one of the pilots and reduced potential project benefits, but project costs were reduced by 20 percent compared with appraisal. The restructuring did not resolve the inherent shortcomings in the project's design discussed earlier that led to low efficiency in project management and high administrative costs for the CCCCC (see ICR, paragraphs 36 and 42). These costs and the substantial lag in disbursements reduced the project's efficiency.

On balance efficiency is rated as modest..

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The relevance of the project's objective was high. Ratings for the relevance of both original and revised project designs were modest. Out of six expected outcomes that contributed to the project’s PDO, five were rated as having substantial efficacy and just one modest, The project's efficacy was therefore rated as substantial. Finally, the project's efficiency was rated as modest.

Based on substantial relevance overall, substantial efficacy together with modest efficiency, there were moderate shortcomings in the project’s outcome. The project's outcome is therefore rated as moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

Outcomes from this project such as a plan for the adaptation to climate change in Dominica's two national parks, rain water harvesting and recycling of waste water in Saint Lucia, as well as desalinization investments in St, Vincent and the Grenadines are all consistent with climate change policies of the participating countries (PCs). It is likely that there will be strong support for sustaining these types of activities and investments in all the PCs. countries.

However, as the ICR points out, farmers may still encroach on national parks and the Caribbean remains highly vulnerable to weather related hazards. Hence improved park management plans could be undermined by severe weather. In addition, small island states participating in this project, are heavily dependent on tourist revenues which fluctuate and could put adaptation programs in the national parks at risk. On balance the risk that development outcomes will not be sustained is moderate.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
The ICR identified shortcomings facing the project at entry namely:

  • The "lack of specific detailed pilot activities; lack of clear national counterparts with the experience to ensure proper execution at the local level; and a core implementing agency that did not have enough control mechanisms to coordinate activities in sovereign countries, led to the restructuring of the Project after the mid-term review" (para 81).
  • In addition, the delegation of responsibility for feasibility studies and detailed pilot project designs to country institutions delayed the project's implementation because country focal points lacked adequate authority to make decisions related to the project’s implementation. These were all significant shortcomings at the project's start.
  • Two pilots were cancelled at restructuring because land tenure and land use policy issues could not be resolved. The Restructuring Paper states that these issues were too serious and complex to be handled during implementation. Land policy issues should have been addressed during appraisal.
In summary, there were significant shortcomings at entry in the project's design, policy matters, as well as institutional and implementation arrangements.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
The ICR noted the sluggish response by the Bank to early implementation constraints (para 84). The delay in taking remedial action compounded the project's problems.

As it involved many stakeholders and governments that required continuous efforts by the Bank team over an extended period, project restructuring was finalized only eight months before the original closing date. In the event restructuring was clearly too late because even at the project's extended closing date a number of significant issues were left unresolved such as the Milton area irrigation development in Dominica, as well as the integrated water supply scheme to manage domestic water supply stresses related to climate change and the water fees issue on Bequia Island in St Vincent and the Grenadines. The substantial delay in the Bank's action to arrange restructuring of the project was a significant shortcoming in the Bank’s supervision
performance.

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
At appraised it was intended that the project's regional executing agency (the recipient of the GEF grant) , the Caribbean Community Climate Change Centre (CCCCC), would depend on national coordinators (appointed by the participating governments and reporting to the CCCCC and the country Ministries of Environment) empowered with the knowledge and experience to ensure planning and execution of the project at the local level (PAD, Annex 6). The ICR highlighted three problems with the arrangements in the PCs, namely (i) national coordinators were not empowered to make crucial decisions on project execution at the local level; (ii) the governments did not meet their financial commitments under the grant; and (iii) national coordinators were not always qualified to support infrastructure projects (paragraph 91). The ICR provides examples of the difficulties faced in Dominica and St Vincent and the Grenadines. In St. Lucia there were fewer problems than in other countries because of the skills of the project coordinator (ICR, paragraphs 81 and 91).

However, despite early difficulties ultimately the three Governments were committed to the project and ultimately results were achieved - with moderate shortcomings.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
At the regional level the implementing agency was the CCCCC which worked constructively under Participation Agreements with national coordinators in the PCs who had weak mandates for taking action . Despite CCCCC's relatively inexperience in the supervision of Bank projects it effectively discharged a range of advisory, monitoring, evaluation and fiduciary functions. Following the reduction of PC funding for the CCCCC because of the impact of the global financial crisis it "made remarkable efforts to step up and cover financial shortages" (ICR, paragraph 94). There was a combination of factors that created difficulties for the CCCCC's facilitation and supervision, as well as the implementation of the pilots and related project activities.

By the project's close the CCCCC had performed all these complex functions with only moderate shortcomings.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The design of M&E was based on the project’s results matrix in the PAD and the revised results matrix in the Restructuring Paper. It would have provided an adequate basis for assessing the project’s progress. The PAD foreshadowed the development of an M&E "guideline and system" in the Operational Manual (paragraph 50), but there is no evidence in the ICR of a dedicated M&E system or capacity in either the CCCCC or in the PCs.

b. M&E Implementation:
In practice the CCCCC was responsible for all M&E implementation but it worked with focal points in the PCs who were required to submit regular monitoring reports. The ICR notes that after restructuring the results framework was simpler and indicators of the project's progress were easier to measure. On the other hand indicators for measuring progress of pilot projects were not always defined. The TTL stated that information on the project's progress was received by the CCCCC from the PCs, but the project closed very soon after most pilots were established and hence there was no time to properly test and improve the M&E arrangements. There was also no budget for surveys and impact evaluation and hence no reports on such matters were prepared (ICR, para 46). Since most activities in this project were pilots, the lack of funds for M&E is surprising.

a. M&E Utilization:
There is no evidence of the utilization of either information gathered during project implementation or any evaluation of that information.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
This was a category B project and triggered action on environmental assessments and natural habitats. The ICR states that Environment Management Plans were prepared for the modest construction of water intake and distribution in the irrigation demonstration project in Dominica, the retrofitting of the Marchand building, for the rain water and waste water treatment plant in St Lucia, as well as for the desalinization plant on Bequia island in St Vincent and the Grenadines. A simplified Environmental Impact Statement was prepared for the desalinization plant.

b. Fiduciary Compliance:
Financial Management: The ICR reported that all financial management requirements during project implementation were met, that financial reporting was timely, and that audits of financial statements were unqualified. A final external audit report concluded that project funds had been used for the purposes intended.

Procurement. Training in Bank procurement procedures was provided at the start of the project. However, procurement was delayed at crucial times because a qualified procurement specialist in the CCCCC resigned. The Project Manager then became responsible for all procurement. This had a detrimental impact on the speed of project implementation during 2009 Nevertheless, procurement plans were kept up to date and post procurement reviews found no problems with the quality of procurement.

c. Unintended Impacts (positive or negative):
The ICR noted that in Dominica the irrigation pilot would be operated by a water users’ association. This was the first water users’ association in the country. Another unexpected benefit of the project was the experience gained through the drafting and signing of a power purchase agreement for the desalinization plant on Bequia island in St Lucia. Finally, the involvement of the private sector in rain water harvesting was also a new initiative for St Lucia.

d. Other:
None



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Moderately Satisfactory
Moderately Unsatisfactory
This Review rated both quality at entry and supervision as moderately unsatisfactory because of the project’s weak design and the delayed action on the project's restructuring to adjust its design. The weak design led to substantial delays during implementation - particularly during the first half of the project’s implementation period. The restructuring came too late - only 8 months before the original closing date. However restructuring streamlined the project, made it simpler and, in conjunction with an extension of the closing date, eventually turned the project around. Nevertheless, this was not in time to avoid a number of project activities being left unfinished at the time of implementation completion. 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR listed a number of lessons derived from the project. They were generally standard list which included the importance of:
  • An institutional capacity assessment and policy framework should be the point of departure to achieve an adequate project design
  • Early engagement with the user community in project planning and design had a positive impact on implementation because their inputs aided the design of and execution of effective pilots
  • Institutional arrangements for implementation need to be documented in an operational manual to ensure effective implementation because it clarifies lines of responsibility for project coordinators
  • Strong engagement of the private sector led to successful implementation and the education of other subsequent stakeholders in the private sector

This Review suggest two other lessons, namely:
  • Bank supervision should be alert at the start of any project to the need to make adjustments in project design as soon as possible and that delays in restructuring a project are likely to be very costly in terms of foregone opportunities for the project.
  • Weak capacity and inadequate authority of country focal points in any regional program can slow down decision-making and implementation and needs to be addressed in the project’s design phase.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is candid and provided a considerable amount of information on which to base its assessment of the project's performance. However a number of the standard annexes and analyses required in the ICR Guidelines were missing undermining the report leaving it only marginally satisfactory.
  • Annex 1 did not provide standard information on project costs and financing, although some of the information normally in Annex 1 was provided in Annex 3 as part of a discussion of the project's restructuring.
  • Annex 2 was a table on "Implications of Climate Change for Economic Activity in Caribbean Countries" whereas the Guidelines require that this annex focus on "Outputs by Components". Some of the information on the project’s achievements and implementation by components that is normally in Annex 4 was instead included in Annex 3. Although the main text provided information on implementation by components, the only annex that came close to a standard Annex 2 was Annex 4 in the ICR which was the revised results matrix agreed for the restructured project, which also included the status of outcome indicators at the project's closing date.
  • Annex 5 titled Economic and Financial Analysis was a summary of Annex 11 in the PAD (Incremental Cost Analysis) and provided no additional information for an incremental cost analysis of the project at completion nor any estimates of rates of return for pilot projects. The availability of economic and financial analyses of the pilots (subsequently by the TTL) was not mentioned in this annex.

    a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions