Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Integrated Management Of Critical Ecosystems Project


  
1. Project Data:   
ICR Review Date Posted:
12/09/2013   
Country:
Rwanda
PROJ ID:
P070700
Appraisal
Actual
Project Name:
Integrated Management Of Critical Ecosystems Project
Project Costs(US $M)
 4.3  3.89
L/C Number:
Loan/Credit (US $M)
 4.3  3.89
Sector Board:
Environment
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  06/30/2005
 
 
Closing Date
10/15/2009 06/30/2011
Sector(s):
Central government administration (40%), Sub-national government administration (32%), General agriculture fishing and forestry sector (20%), Other social services (5%), Agricultural extension and research (3%)
Theme(s):
Participation and civic engagement (25% - P) Biodiversity (25% - P) Environmental policies and institutions (24% - P) Land administration and management (13% - S) Water resource management (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Rimma Dankova
Ridley Nelson Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:


    This GEF project was a component of the first phase of the Rural Sector Support Program (RSSP), a 14-year three-phase Adaptable Lending Program. However, midway through implementation it was delinked from the RSSP upon the completion of that project. The development objectives of the first-phase were to equip farmers, other target private sector operator groups, and the relevant government institutions with the basic institutional and technical capacities that should lay the groundwork for subsequent productivity-raising interventions in the areas of: (a) agricultural services delivery systems; (b) small-scale rural infrastructure development and maintenance; (c) rehabilitation of marshland and hill-side farming; (d) promotion of traditional and alternative export agriculture, and (e) diversification of economic activities in the off-farm sector of the rural areas.

    The specific objectives of this GEF-financed project (Project Appraisal Document, page 2) are:

    “to help the farmers to adopt sustainable agricultural intensification technologies that increase agricultural productivity and improve livelihood while protecting the natural resource base.”

    The statement of objectives in the GEF Grant Agreement (page 20) is effectively identical, but adding the means:

    “to assist farmers to adopt sustainable agricultural intensification technologies that increase productivity and improve livelihood while protecting the natural resource base, through the development and implementation of community-based integrated ecosystem management plans, using watershed and micro-catchment areas as primary units for resource planning.”

    This review uses this Grant Agreement statement of objectives as the basis for assessing the project.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

1: Development of a policy and regulatory framework for sustainable wetland and natural resource management (estimated cost at appraisal US$0.30 million; actual cost US$0.47 million).This component aimed to help develop a sound policy and institutional framework, including the development and adoption of regulations and legislation related to the use and protection of wetlands, and the institutional arrangements required to support the integrated management of critical wetland ecosystems. It also aimed to promote and facilitate inter-ministerial coordination in natural resource management, support the integration of conservation and biodiversity aspects into sectoral policies and programs, and support implementation of the government’s decentralization policy in areas related to decentralized natural resource management. It was to do so by helping to establish effective coordination and oversight mechanisms at the local government level to support the implementation of community-based integrated ecosystem management at the watershed and wetland catchment level. Finally,this component was to promote participation and collaboration in joint management of natural resources among a wide range of stakeholders, including the public sector, NGOs, private sector, and resource users.

    2: Capacity building and institution strengthening for integrated ecosystem management (estimated cost at appraisal US$1.50 million; actual cost US$1.13 million). This aimed to build a critical mass of trained individuals at the central, regional, and local levels who would help the design, implementation and monitoring of integrated resource management plans associated with the rehabilitation of wetland for agricultural production, and the protection of biodiversity in critical wetland systems. The training and capacity building program was to include direct technical assistance, workshops, on-the-job training, and degree training, and empowerment of local communities in natural resource management through technical capacity strengthening. The identification, planning, and implementation of these capacity-building activities was expected to be closely coordinated with those of the Rural Sector Support Program, and collaborative training and environmental awareness-raising activities were also to be developed with assistance of the UNDP GEF Protected Areas Project, and the African Development Bank Environmental Management Institutional Support project. In addition, an Environmental Information System (EIS) was to be developed to facilitate the generation and the flow of environmental information and data for decision-makers, resource users, the scientific community, and the wider public. Priority was to be given to the development of a Biodiversity Information System. It was expected that the project would finance the costs of biodiversity assessment of the four larger wetlands and the establishment of mechanisms for the participatory monitoring of environmental management and change in the project sites. The knowledge thus generated would be used (i) to develop a National Strategy and Action Plan for the conservation and wise use of wetlands;and (ii) to help incorporate biodiversity considerations into the management plans of the individual watersheds and wetlands, and into sectoral investment programs.

    3: Development and implementation of community-based IEM plans for critical ecosystems (estimated cost at appraisal US$1.70 million; actual costs US$1.49 million). This component was to be the main channel through which the benefits of the enabling conditions created under components 1 and 2 would be expected to materialize. Community-based integrated ecosystem management plans would be developed and implemented to help integrate soil, water, and biodiversity conservation into the rehabilitation of the wetlands for agricultural production. In addition to helping farmers adopt technologies that help sustain the agricultural resource base, this component would select and protect four globally important and threatened critical ecosystems and habitats associated with major wetland systems. Catchments and watersheds would be the spatial unit for diagnosing ecosystem management problems, and for planning interventions. The watershed management plan was to address cross-ecosystem linkages, including upstream-downstream interactions, and hilly terrain, and integrate economic and social dimensions (population density, social structure, livelihood) into the conservation plans for land, water, forest and other natural resources. Watershed and community-based integrated ecosystem management plans would concentrate on in situconservation activities and be developed using a participatory and iterative process.
    4: Project management, monitoring and evaluation, and information dissemination (estimated cost at appraisal US$0.80 million; actual US$0.85 million).This component supported activities to: (a) ensure effective coordination of the project implementation activities; (b) monitor and evaluate these activities; and (c) disseminate lessons learned from targeted research, and promising ecosystem management practices.

    d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

    Project Cost:
    At closing, the project total cost was US$ 3.94 million, slightly less than the appraisal forecast (US$ 4.30 million) because of underspending on components 2 and 3.

    Financing:
    The project financing consisted of a GEF Grant of US$ 4.30 million Actual disbursement was US$3.94 million. US$0.40 million was cancelled at project closing..

    Borrower Contribution:
    The Government of Rwanda actually contributed US$0.91 million compared with the appraisal estimate of US$ 1.00 million.

    Dates:
    The GEF project was originally designed as part of the blended IDA/GEF Rural Sector Support Project, but it became a stand-alone project on December 31, 2008 when the RSSP closed. At that time only 46% of project funds had been committed, as its original closing date was October 15, 2009. Subsequently, the project closing date was extended twice. The first extension from October 15, 2009 to December 31, 2010 provided for a phased completion of priority activities within all components, and monitoring and reporting. The second extension was by 6 months from December 31, 2010 to June 30, 2011 to complete all intended activities. Final reallocation of proceeds across categories was done June 30, 2011 to bring the project to financial closure.


    3. Relevance of Objectives & Design:

    a. Relevance of Objectives:
    Rated Substantial


    The project objective was substantially relevant at appraisal and currently. The expanded irrigated area in cultivated marshlands and associated hillsides was expected to accelerate the pace of agricultural intensification. However the growing population (one of the densest in Africa) and poverty has created increasing pressure for rural populations to cultivate unsustainably on hillsides and in wetlands. The resulting land degradation and wetland conversion is threatening land productivity, water sources, and Rwanda’s rich biodiversity. Raising agricultural production in a sustainable way is one of the directives of the Government of Rwanda Economic Development and Poverty Reduction Strategy for 2008-2012 and Rwanda Vision 2020. The project was also highly relevant to priorities of the Bank’s Country Assistance Strategy for Rwanda in 2009-2012 "to promote the sustainable use and management of Rwanda’s critical ecosystems, particularly marshlands".

    Rwanda’s diversified landscape has several wetlands that are important from a biodiversity perspective, including an internationally recognized Ramsar site that is home for endangered and rare bird species. Project objectives were relevant to GEF OP12 integrated ecosystem management objective to “achieve multiple national and global benefits by promoting the widespread adoption of farming practices that integrate ecological, economic, and social goals and reduce land and wetland ecosystems degradation.”

    b. Relevance of Design:
    Rated Modest.

    The composition of project components was relevant to the achievement of the project objectives. The GEF incremental financing aimed at providing technical, institutional, and financial support to farmers in order to help them make the transition from traditional farming practices to improved technologies for increased food production and biodiversity conservation outside protected areas. This was to be achieved through activities to: (i) design a holistic approach to addressing the resource constraint facing farmers; (ii) develop a sound policy and institutional framework for natural resource management both at the central and local levels; and, (iii) remove the technical, institutional, and financial constraints to integrating the conservation of biodiversity, water, soil fertility, and the protection of watersheds into farming practices (PAD, page 9). The development and implementation of community-based integrated ecosystem management plans using the watershed and micro-catchment areas as the primary units for resource planning, was instrumental for integrating soil, water, and biodiversity conservation into the rehabilitation of the wetlands for agricultural production. The project outputs were also to include compatible on- and off-farm incentive systems to be in place to support implementation of watershed and community-based integrated ecosystem management plans (PAD, page 11).

    However, the project design underestimated the implications of limited implementation capacity in the newly forming institutional setting of the Government of Rwanda by designing an implementation period of only 4 years. The original project scope was too ambitious, addressing multiple issues (biodiversity, land degradation, carbon sequestration, agriculture productivity, poverty reduction, etc.) with limited financial resources and institutional capacity. The proposed project was thought to bridge the implementation capacity gap through a concerted effort with the on-going RSSP project. In spite of this, project design had a significant shortcoming in defining how the partially blended RSSP and GEF projects would coordinate their activities and provide mutual synergy.

    The result framework was largely logical but not enough consideration was given to how some outcomes would be measured (for example, the outcome on biodiversity increase). Also, the framework missed including outputs on putting in place incentive systems for farmers to implement integrated ecosystem management plans.


    4. Achievement of Objectives (Efficacy) :

    “To assist farmers to adopt sustainable agricultural intensification technologies that increase productivity and improve livelihood while protecting the natural resource base”.


    The statement of the project objective has three sub-objectives: “to (a) help the farmers to adopt sustainable agricultural intensification technologies; (b) increase agricultural productivity and improve livelihood; and (c) protect the natural resource base. The level of achievement of these objectives is assessed below under the three sub-objectives.

    (a) Assist farmers to adopt sustainable agricultural intensification technologies: Rated Substantial

    Under this heading predominantly output or intermediate outcome level achievements are reported. The agricultural outcomes are covered under heading (b) and natural resource outcomes under heading (c).

    Capacity and adoption of sustainable agriculture

    The project jointly with RSSP contributed to increasing the capacity of Farmers Agricultural Services Delivery Organizations (FASDOs) to deliver research and extension services, and the capacity of farmers’ associations and cooperatives and government institutions to adopt sustainable agricultural practices. Support was also provided to rehabilitation of small-scale rural infrastructure, to rural micro enterprises, and program management and implementation.

    The capacity of FASDOs in hillside protection and management was strengthened with 5,365 FASDOs trained. The impact assessment field survey (2008) found that 75 percent of FASDOs were trained in agro-forestry measures, 16.2 percent in terracing and 11.7 percent in soil protection (ICR of the RSSP project, paragraph 69).

    A total of 6,832 farmers were trained in better management of their crops, input use and IPM practices (4,166 men, 2,666 women)--almost 10 times more than the agreed target. The achievements were considerably higher than the baseline and well above the set targets.

    Farmer associations improved management of their finances, production, infrastructure, and marketing, including the improved capacity of 23 market management committees in management and maintenance of marshland infrastructure.

    A demonstration investment in radical terracing and land tenure provision was undertaken in the Burera district to reduce pressure on the Rugezi wetlands.

    Adoption of Integrated Ecosystem Management (IEM)

    Community-based Management Plans were prepared in nine critical wetlands in the targeted under the project ecosystems, exceeding the planned outcome of four such plans envisaged at appraisal.

    District Development Plans for IEM were developed in four districts, which is reasonably consistent with the target of half of the districts (ten in total) involved in the project.

    Water Management Committees (WAMACOs) were established in 53 sectors (subdivisions of districts) to develop and implement watershed and wetland management plans.

    A range of training materials, DVDs, and various publications were developed for distribution to schools and the general public. Numerous training activities for Integrated Ecosystem Management were provided to 159 WAMACO members, including training for soil and water conservation, tree nurseries, and ecotourism.

    Study tours were organized and farmers were trained on several issues including soils, land and water management and water quality testing.

    Small works projects demonstrated the range of wetland ecosystem benefits. Twenty-four spring sources were upgraded, improving water supply and quality to a few thousand people. The water sources were protected with concrete casements and basic upstream protection, and communities trained on water quality testing.

    Eight watchtowers were constructed to improve community and visitor appreciation of the wetland biodiversity, and to catalyze ecotourism in the area.

    Other community-level activities included establishment of tree nurseries in 42 sectors, and planting of Pennisetum (elephant grass) to protect boundaries of marshlands in 40 locations.

    (b) Increased agricultural productivity: Rated Substantial

    Much of the outcome level achievement of the project comes under this heading.

    Achievements reported below are the results of the complementary efforts of the phase-one Rural Sector Support and the Integrated Ecosystem Management projects implemented jointly. There is therefore only partial attribution. It was not possible to convincingly separate the impacts of the two projects.

    Increased Agricultural Productivity

    121,366 direct beneficiary farmers reported an increase of over 50% in their rice production during the last two seasons of project implementation.

    Harvest values were surveyed to be significantly higher for hillside as well as marshland farmers.The surveys conducted under the RSSP project indicate that incomes in both marshland and hillside areas rose substantially. Direct project beneficiaries reported increased market sales of 50% (compared to 15% for non-beneficiaries), hillside harvest value increased by 14% and marshland harvest value increased by 25 times over non-beneficiaries.

    There was technical and financial support provided to farmers to implement sustainable agriculture practices, including terracing of the 120 ha of lands and agro-forestry. Potential yield gains were estimated to be in the order of 126 tons/yr for maize; and 720 tons/year of potatoes.

    The project contributed to the reported outcomes of RSSP on rehabilitation and development of about 3000 ha of marshlands and almost 15000 ha of hillsides. Improved techniques were adopted in rice cultivation in newly developed irrigated marshland schemes. Fertilizer use also increased significantly.


    (c) Protect the natural resource base. Rated Modest

    The biodiversity and NRM intermediate outcomes and outcomes are covered under this heading.

    Policy and institutional framework for wetlands development and implementation.

    Ministerial orders were adopted in 2009 to establish the first policy framework for wetlands management in the entire country, by categorizing wetlands and determining their protection and use status. Marshland decrees drafted and adopted under Land Law and National Marshland Law was drafted and expected to be approved in 2012.

    The Project had originally envisioned the development of a National Strategy and Action Plan for Wetlands however these two activities were dropped at the Mid Term Review. It was recommended that the Strategy and Action Plan would be a post-project activity that could benefit from the project results in improved biodiversity knowledge base and build upon experience in developing and implementing the first policy documents mentioned above.

    An Organic Law on Environment(2005) was passed by the Government that sets and regulates activity in 50-meter buffer zones for wetlands, lakes, and rivers.

    Institutional capacity of Rwanda Environmental Management Agency (REMA) was strengthened at a critical formative stage. Through the project, REMA built its institutional and technical capacity at national and decentralized levels. National, district and local authorities were sensitized regarding the importance of wetlands and the need for the integrated ecosystem management approach

    REMA promoted inter-sectoral coordination on wetland protection by participating actively in broader alternative mechanisms such as discussions in Sector-wide Clusters and annual “National Dialogues”, which are active in addressing national environmental issues. Additionally, REMA facilitated the placement of environmental specialists in other ministries (e.g. Finance and Economic Planning, Local Government, Agriculture, Industry and Trade, Infrastructure, and Environment and Natural Resources) to provide mainstreamed advice on environmental laws, regulations, and issues. However, a permanent mechanism for inter-ministerial coordination for integrated ecosystem management and wetlands protection was not implemented as envisaged at appraisal.

    Wetland Management Knowledge base.

    Through the project, all Rwandan wetlands were mapped and classified. A climate change vulnerability assessment on these wetlands was conducted to examine the impact and adaptation needs of changing climate patterns in Rwanda.

    A Biodiversity Catalogue was initiated to catalogue all key flora and fauna so far identified in Rwanda. Wide consultations were held on biodiversity indicator development, and a Biodiversity Information System platform was established. The first detailed inventory (vegetation, birds, mammals, reptiles, and amphibians) of the four most important wetland ecosystems in Rwanda has been created with the help of international expertise. A Biodiversity Atlas was developed for these ecosystems.

    Protection of the natural resource base.

    All wetlands were legally categorized for protection and use potential through the project. The ICR reports that uncontrolled conversion of wetlands to agricultural production has ceased as a result of the introduced requirement for environmental assessment of wetlands prior to conversion decisions. According to the ICR, wetlands of global importance are fully protected, and cannot be converted to other uses.

    Improved land and water management practices are adopted in 80% of the wetlands rehabilitated by the RSSP.

    Finally, of most significance to the protection of natural resource base rating, there is limited evidence of outcome achievement. Targets were set at appraisal to achieve a stability or increase in two to three indicators of biodiversity (fauna, flora to be determined in a baseline survey) in four major wetland ecosystems. However, while there are a number of outputs and intermediate outcomes including legislative and capacity achievements with some attribution to the project, there are no reports of key indicators that would demonstrate that the biodiversity outcome targets were achieved.


    5. Efficiency:

    There was no estimation of the Economic or Financial Rates of Return at appraisal and at closure. However, the ICR (page 34) quantifies some project benefits. This includes agriculture gains resulting from sustainable land management activities on hillsides (to reduce erosion and protect wetlands) with an estimated present value of US$1.3 million (at a 10% discount rate). The increased agricultural output was spread over a considerable number of households and, according to the ICR, benefited at least 2,000 people. The ICR (page 34) also estimates that about 4,000 people have benefited from the improved water access resulted from spring sources upgraded under the project. The ICR also reports improved water flow from the protected Rugezi wetlands that created an economic benefit of restored hydropower generation estimated at US$10 million a year. However the attribution of this benefit to the project is questionable since the increase in water flows may be due to the wetter conditions observed in that period of time.

    The project achieved substantial efficacy on two out of three project development objectives (see Section 4). Project long-term benefits were substantial given the low cost of the investment at about US$4 million. This included contribution to policy and the legal framework, strengthening of the Rwanda Environmental Management Agency as an important advocate for wetlands and a demonstrated Integrated Ecosystem Management process at community level. Although it took longer than planned to implement the project, administrative costs were only marginally higher than estimated at appraisal.

    Overall, efficiency is rated substantial.

    a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


    Rate Available?
    Point Value
    Coverage/Scope*
    Appraisal:
    %
    %
    ICR estimate:
    %
    %

    * Refers to percent of total project cost for which ERR/FRR was calculated

    6. Outcome:


    The relevance of project objectives was substantial. Relevance of project design had significant shortcomings that led to delays in project implementation. There was only partial achievement of some expected outcomes but achievement of two of the three project objectives is rated substantial with significant agriculture outcomes albeit only partially attributable. There was insufficient evidence on biodiversity outcomes despite a number of important outputs and intermediate outcomes in natural resource protection. Efficiency is rated substantial on the grounds of significant achievements for modest expenditure.

    a. Outcome Rating: Moderately Satisfactory

    7. Rationale for Risk to Development Outcome Rating:

    The increased institutional capacity has enabled Rwanda Environmental Management Agency (REMA) to become a strong advocate of wetland protection and biodiversity conservation. It also intends to continue supporting the developed knowledge base, including a biodiversity catalogue and the biodiversity information system, and hired additional staff for that purpose.

    The Ministry of Natural Resources and REMA are planning to upscale the project initiatives through the Lake Victoria Environmental Management Project II that includes Rwanda; a proposed GEF Rwanda Forest Landscape Restoration project; and various climate change related initiatives.

    The institutional structures for environment and natural resources management, including the Environmental Management Agency, are fairly new in Rwanda. The Environment Law had only been approved in 2005. While a sound policy and institutional framework for wetlands management was developed under the project, a clear enforcement mechanism is still lacking to support its implementation. The project also was unable to establish a permanent inter-sectoral coordination mechanism within the government for effective wetlands management. This, along with the absence of a National Wetland Strategy and Action Plan, creates significant risks to sustainability of the projects results.

    The project was largely successful in demonstrating to farmers the benefits of sustainable agriculture intensification combined with ecosystem management. Integrated Ecosystem Management (IEM) plans were prepared at community levels. However there are risks to the implementation of these plans if no effective incentive systems are put in place to encourage farmers to adopt the IEM approaches.

    There are risks associated with technological change and climate. If REMA cannot keep its information systems up-to-date with changing technologies, Biodiversity Information System platform, biodiversity knowledge base and its usefulness will be compromised.

    If extreme weather events displace populations, threaten water supplies, and endanger human health, pressures on ecosystems may become unmanageable.

    a. Risk to Development Outcome Rating: Significant

    8. Assessment of Bank Performance:

    a. Quality at entry:

    Project preparation correctly identified the four critical ecosystems of Rwanda as the project’s focus in managing biodiversity in non-protected areas and productive landscapes. However, there were a number of serious shortcomings during the very long 5-year preparation period. The linkage with the RSSP was not operationally well defined and the risk of poor coordination between the projects was underestimated. The results framework was weak and selected key performance indicators were significantly inconsistent across project documents. The suggested monitoring framework was weak. Critical institutional weaknesses were overlooked at the design stage. Greater attention to early adequate staffing of REMA’s project management team could have improved technical, fiduciary and procurement performance. The detailed design and costing of some of the activities (especially the biodiversity information system) would have benefitted from additional preparation.

    Quality-at-Entry Rating: Moderately Unsatisfactory

    b. Quality of supervision:

    There were three different Task Team Leaders over the life of the project that also affected the project’s institutional memory and consistency of supervision. In the initial years of project implementation, quality of Bank supervision was variable and complicated by poor coordination with other components of the Rural Sector Support project (RSSP). Bank implementation support improved after the project Mid-Term Review when the project was delinked from the RSSP. This, together with clarified key performance indicators, an extended implementation timeframe and reallocated project resources across categories, provided a better framework for project supervision. Placement in Kigali of a senior Bank biodiversity specialist to specifically support project implementation also helped to improve quality of supervision. Following these adjustments, the project performance rating changed from moderately unsatisfactory to moderately satisfactory.

    Quality of Supervision Rating: Moderately Satisfactory

    Overall Bank Performance Rating: Moderately Unsatisfactory

    9. Assessment of Borrower Performance:

    a. Government Performance:

    The Government provided strong support to establish an effective and pragmatic policy and legal framework for ecosystem and natural resources management in the country, including support to wetlands conservation and management. It was also committed to promoting decentralized and participation of project stakeholders. A major shortcoming in the government performance was the lack of an effective partnership with the Ministry of Agriculture that resulted in a missed opportunity for the project to improve inter-sectoral coordination over integrated ecosystem management. The ICR also notes that Government could have made a better contribution to the development and implementation of an effective M&E system.

    Government Performance Rating: Moderately Satisfactory

    b. Implementing Agency Performance:

    The project was implemented by the Rwanda Environmental Management Agency that, according to the ICR (page 20), demonstrated strong commitment and leadership. Over the course of project implementation, the Agency gradually improved its staffing and outreach to other ministries and assumed leadership to further improve the policy and regulatory framework for environmental management in the country. The Agency, however, faced a number of issues related to inadequate staffing. Not much effort was made to replace a full-time project coordinator and qualified technical staff, who left the project in 2010. Not replacing project-dedicated staff on a full-time basis adversely affected the project implementation quality. High turnover of the project’s procurement and financial management staff also contributed to delayed procurements and slowed disbursements. The PMU demonstrated shortcomings in project reporting and M&E.

    Implementation performance dramatically improved after the project was delinked from the RSSP in 2008. The PMU was given full responsibility for the project, new staff was appointed, and an implementation framework for IMCE’s remaining activities was developed. REMA was able, with Bank support, to reasonably complete the remaining activities despite uncertainties about project extensions and incomplete staffing. Partnerships were established with local universities and international entities such as IUCN (Ramsar) and Conservation International, and young professionals were hired to complete the biodiversity cataloguing.

    Implementing Agency Performance Rating: Moderately Satisfactory

    Overall Borrower Performance Rating: Moderately Satisfactory

    10. M&E Design, Implementation, & Utilization:

    a. M&E Design:

    M&E design did not provide a consistent framework for the project monitoring and evaluation. An inconsistent set of key performance indicators was suggested across different project documents. Some of these indicators were very ambitious and, thus, difficult to measure; e.g., tracking changes in the number of flora/fauna species without having baseline information.

    The results framework was not fully adequate and realistic on a number of outcomes, including such key performance indicator (KPI) as a National Wetland Policy "prepared and approved by the end of project first year, and implemented by end of project year 2" (ICR, Annex 2, page 25). The formulation of KPI on observed changes in flora / fauna resulting from project activities was also vague, particularly without clear identification of relevant species linked to land use changes. There were no indicators to track the collaboration between RSSP activities (overseen by the Ministry of Agriculture and Animal Resources) and those of the project (overseen by Rwanda Environmental Management Agency).

    b. M&E Implementation:

    The M&E indicators were revised at the Mid Term Review when the project was delinked from the RSSP. They became more realistic and directly related to the project activities. However the M&E implementation was not systematic and there was no full-time M&E officer in the PMU, despite the agreement at appraisal. The project Implementation Supervision Reports completed after the Mid Term Review, provided useful information on the various project indicators. Biodiversity indicators were developed, a wetland inventory was conducted, and a Biodiversity Information System was established to track biodiversity and natural resource use trends. A wetland inventory was conducted in the early stage of the project.

    a. M&E Utilization:

    Data collected through the wetland inventory was utilized for the nation-wide mapping of the wetlands. This formed the basis for the country’s policy and legislative framework for the categorization and protection/use of its wetlands. The generated knowledge and the tracking of project activity performance formed the basis of many of the Agency’s awareness-building activities. The Biodiversity Information System, with associated GIS information and biodiversity cataloguing, was developed only towards the final stages of project implementation, and was thus not used as effectively as planned for decision-making during the project implementation.

    M&E Quality Rating: Modest

    11. Other Issues:

    a. Safeguards:

    The project was initially part of the RSSP that was rated category ‘B’ under OP4.01 Environmental Assessment (OP4.01). OP4.09 Pest Management (OP4.09) was invoked. The ICR indicates that an environmental assessment was conducted and several provisions were made to ensure compliance (ICR, page 9). The ICR also reports that community-based small works on “radical” terraces for erosion management implemented under the project were consistent with the RSSP safeguards in this regard. When the project became independent from the RSSP, the issue of potential negative environmental and social impacts was less relevant because very few construction activities were to be implemented.

    b. Fiduciary Compliance:

    Financial Management and Audits: The ICR reports that the financial management and internal control arrangements were weak from inception due to the low capacity of Agency staff. At that time, maintenance of books of account and reporting was inadequate due to the lack of qualified accountants and poorly programmed accounting software. After the Mid Term Review, financial management performance improved as a result of improved accounting software, increased stability in staffing, and assigning an Agency accountant to the project under the supervision of its Finance Director. Interim financial reports were submitted to the Bank on a quarterly basis and annual audit reports were conducted in accordance with international standards.

    Procurement:Procurement activity was slow to start. High staff turnover and the lack of a dedicated full-time procurement specialist at the beginning of the project caused frequent delays in critical procurement and adversely affected project implementation schedules. After the Mid term Review and the recruitment of a Technical Specialist, project procurement advanced according to the procurement plans. The Bank team worked closely with the Agency to improve the understanding of Bank procurement procedures and develop its procurement capacity. A short-term procurement consultant was hired in the final months of the project to support the final set of procurement activities.

    c. Unintended Impacts (positive or negative):
    None reported.

    d. Other:



    12. Ratings:

    ICR
    IEG Review
    Reason for Disagreement/Comments
    Outcome:
    Moderately Satisfactory
    Moderately Satisfactory
     
    Risk to Development Outcome:
    Moderate
    Significant
    Significant risks due to absence of inter-sectoral coordination mechanism and an incentive system for farmers to actually implement the plans. 
    Bank Performance:
    Moderately Unsatisfactory
    Moderately Unsatisfactory
     
    Borrower Performance:
    Moderately Satisfactory
    Moderately Satisfactory
     
    Quality of ICR:
     
    Satisfactory
     
    NOTES:
    - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
    - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

    13. Lessons:
    The following lessons are taken from the ICR with some adaptation:

    1. A small but focused project can make a significant contribution to a national development agenda. The experience of the IMCE project illustrates how a small investment of about US$ 4 million has made a real difference by supporting development of a policy, institutional, and knowledge framework for conservation of critical wetland ecosystem across the entire country; improving public awareness; introducing new paradigms for Integrated Ecosystem Management; and identifying potential areas for new investments in wetland areas.

    2. A long-term biodiversity conservation program relies on keeping information and data bases current. The development of an appropriate knowledge base in terms of biodiversity cataloging and associated information systems should be considered part of the strategy and a prerequisite for creating a basis for awareness raising and investment planning. It is critical that such information resides in the public domain. A new generation of documentation and innovative tools (mobile applications, web portals) can be used to improve access to and use of such information.

    14. Assessment Recommended?

    No

    15. Comments on Quality of ICR:

    The ICR is written in a concise manner, internally consistent and contains most of the elements necessary to evaluate the project. It is informative enough in describing and analyzing the project achievements and limitations and is open about the aspects that did not progress well. Annex 2 is informative and complements the main text with useful details on key performance indicators across different project documents.

    a. Quality of ICR Rating: Satisfactory

    (ICRR-Rev6INV-Jun-2011)
    © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions