Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Gulf Of Gabes Marine And Coastal Resources Protection Project


  
1. Project Data:   
ICR Review Date Posted:
06/09/2014   
Country:
Tunisia
PROJ ID:
P069460
Appraisal
Actual
Project Name:
Gulf Of Gabes Marine And Coastal Resources Protection Project
Project Costs(US $M)
 9.81  7.75
L/C Number:
Loan/Credit (US $M)
 6.31  4.47
Sector Board:
Environment
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  03/10/2005
 
 
Closing Date
06/30/2010 12/31/2012
Sector(s):
General water sanitation and flood protection sector (50%), General agriculture fishing and forestry sector (30%), General public administration sector (20%)
Theme(s):
Biodiversity (29% - P) Participation and civic engagement (29% - P) Water resource management (14% - S) Environmental policies and institutions (14% - S) Administrative and civil service reform (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Keith Robert A. Oblitas
John R. Eriksson Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:


    To establish a functional integrated monitoring and participatory management system for the project area to manage biodiversity degradation in the Gulf of Gabes region.

    Source: PAD. (The GEF Trust Fund Grant Agreement is substantively the same but, by omitting the word “management” after “participatory”, is less precise.)

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

1. Institutional Strengthening, Strategic Planning and Dissemination. (Estimated Cost (base cost without contingencies) at Appraisal: $3.94 million; Actual Cost: $4.32 million)
Creating and operating a Project Management Unit, and Project Operational Unit; quality control and valuation for project activities; and preparing a long-term strategy to protect biodiversity.

2. Training and Capacity Building.
(Estimated Cost (base cost without contingencies) at Appraisal: $1.35 million; Actual Cost: $0.70 million)
Training and capacity building for agency personnel in management, technical, scientific and public participation skills, and in marine and coastal regulations; public awareness for local communities; socio-economic surveys; and preparation of participatory methodologies for local development committees.

3. Baseline Marine Data Acquisition and Applied Biodiversity Monitoring.
(Estimated Cost (base cost without contingencies) at Appraisal: $1.20 million; Actual Cost: $0.89 million)
Acquiring and monitoring scientific data for biodiversity management plans, including hydrodynamic and water quality data, and inventories of marine and lagoon fish species, and alien species, of regional and global interest; development of a regional management strategy for ballast water disposal and dealing with alien species; and studying and recommending changes to fishing practices to sustain biodiversity.

4. Participatory Biodiversity Management Plans.
(Estimated Cost (base cost without contingencies) at Appraisal: $3.32 million; Actual Cost: $1.84 million)
Preparing sustainable biodiversity management plans for six pilot sites and implementing the plans on three of the sites, using participatory processes; preparing, implementing and monitoring a management plan for the sea grass area; inventories and mapping of marine plant cover; and development of a geographic information system as a data base for an Information Exchange Center.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

The project was approved on 03/10/2005 and was planned to close 5 years later on 06/30/2010. Closing was later extended by 2 years to 12/31/2012, primarily to make up for a period of social unrest which set project implementation back by about 18 months. This resulted in a revised total project period (Board to Closure) of 7 years. The project received a Global Environment Facility Grant of $6.31 million of which $4.47 million was disbursed. There were no co-financers. Project costs were estimated at appraisal at $9.81 million, and actual project costs at closure were $7.75 million, 79 percent of the appraisal estimate. Savings came mainly from reductions in the infrastructure on the biodiversity conservation sites due to delays in preparation of the management plans, and possible overestimation of costs at appraisal for other project components. Project objectives were not changed and there was no significant restructuring of project components, although there was some reallocation of funds from consultancies to capacity building activities.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

The project’s objectives responded well to the need to stem a declining marine ecology and a loss of biodiversity in the Gulf of Gabes due to multiple threats on the marine environment including from overfishing, trawling damage, and urban, industrial and ballast pollution. Tunisia lacked both institutional and technical capacity to tackle these problems. Thus, the focus of the objectives on creation of a functional institutional base and of the technical capacity to monitor and manage the marine ecology, was a good first step towards implementing comprehensive conservation actions.

The objectives were also in line with Bank, GEF and Government strategy. For GEF, biodiversity conservation is a central pillar in its Global Environment Objectives. For the Bank, the November 2009 Country Partnership Strategy referred to improving and conserving natural resources, and specifically to better management of coastal and marine natural resources. Project objectives also reflected Government’s Strategy for Protection of Marine and Coastal Resources in the Gulf of Gabes (2004) and its Procedures for Participation of Communities in the Management and Conservation of Marine and Coastal Resources in the Gulf of Gabes (also 2004). The Relevance of the project’s Objectives was High.

b. Relevance of Design:
Note: In two areas – institutional arrangements and stakeholder participation – there are some inconsistencies in viewpoint between the ICR and the views of the Task Team Leaders during appraisal and project implementation who were interviewed by IEG. The ICR, as the official document of the Region, is the authoritative source for this ICR Review. However, where comments from TTLs help clarify or are significantly different from the ICR, such comments have been noted. For efficiency, most of these comments have been placed in this section, as below:

The Results Framework (PAD, Annex 1) provides a logical connection between the components and their detailed features, and the project objectives. Design contained the technical measures to monitor and implement biodiversity plans, which required a sophisticated scientific base. Also appropriate was the substantial training included in the project, which recognized Tunisia’s limited capacity managing biodiversity. Hinging the project on two thrusts – capacity building and the introduction of monitoring and planning technologies – encompassed most of the actions required and was strategically relevant.

Participatory approaches were also highlighted, but the participation intention was less well translated to a course of action that would achieve this. For instance, the PAD’s description of component 4 (“Preparation of the Participatory Biodiversity Management Plans”) on page 7 and in the “Detailed Project Description” in Annex 2 contain very limited discussion of the processes by which communities would be motivated, trained and provided technical assistance in forming participatory processes and institutions. There is sparse elaboration also in the depth and breadth of the stakeholder communities (e.g. leaders only? village meetings? all stakeholders interests such as commercial, urban, tourism, villages?). different types of fishing interests such as offshore trawling, village fishermen; and the interests and participation of women and the disadvantaged). And the ICR comments, referring to the project’s outcome as a whole (page 8), that there were “some shortcomings in terms of the adoption of truly participatory approaches.” Given the PAD’s shortfalls, and the participatory weaknesses discussed a number of times in the ICR, participatory processes could have been further developed in project design (and in project implementation – refer section 4). (NB. Three Task Team Leaders of the project - at appraisal, early implementation and at the end of the project - expressed to IEG that they considered that adequate participatory processes were employed.)

There are also issues related to the complexity of the project and the institutional arrangements for implementation, which the ICR describes as lacking in the degree of integration and cooperation between agencies. There were four implementing entities (the General Directorate of Environment and Quality of Life responsible for Institutional Strengthening, Strategic Planning and Dissemination, component 1); the International Center for Technology and the Environment of Tunis, responsible for Training and Capacity Building, Component 2; the National Institute of Sciences and Marine Technologies, which implemented most of the environmental monitoring and technical studies, Component 3; and the Coastal Planning and Protection Agency responsible for the participatory management plans, component 4). It was a sensible decision to leave institutions with their existing mandates, providing that institutional mechanisms were created to coordinate and integrate activities. To accommodate this, a Project Steering Committee was established. There was also at more operational level a centrally based Project Management Unit, and a Project Operational Unit at Gabes. The Operational Unit included full-time representatives from all four implementing agencies.

In principle, the Committee, PMU and POU might have been an adequate bridging structure. But the ICR advises that they were not, and that during implementation there was insufficient integration and coordination of activities: “The four components were managed by the four implementing agencies as quasi-autonomous ‘sub-projects’ with minimal interactions (each component/agency having its own budget, specific procurement procedures , action plans, chronograms, and hierarchical reporting).” (ICR, page 7); “the Project Steering Committee was not efficient because of too many participating institutions, inadequate knowledge of the project by its participants, and rarity of meetings. The Steering Committee therefore failed to act as a multi-sectoral institution providing overall policy guidance to the project, and acted sporadically only as a simple network to exchange and discuss information and define potential synergies.” (ICR, page 7); and “Each implementing agency was and is still operating in isolation with minimal interaction.” (ICR, page 19). By contrast, the TTLs consulted by IEG, while acknowledging that project coordination was a challenge, felt that the institutional structure as designed worked better for project coordination than indicated in the ICR. Coordination, may, however, not be enough. Environmental conservation needs an integrated approach, all elements informing, interacting and cooperating with the other elements. An exit strategy was also needed for when the project ended, together with the funds for the entire coordination structure that it created. But, as advised in the ICR (page 12), “Implementation arrangements ………. did not set a model for replication.” (Although a long-term strategy for biodiversity management was envisaged at appraisal and included amongst other matters, consideration of institutional mandates (PAD, page 21).

In summary, the project’s design, while technically strong, had two weaknesses: It did not provide sufficient implementation details to fully achieve the objective for holistic participatory management or to define what degree of participation was intended. And second, while the institutional structure was at least partially effective as concerns project coordination, the arrangements were not conducive for development of an integrated system for managing biodiversity, and the arrangements did not articulate an exit strategy, other than what might have emerged in a study of bio-diversity strategy. The Relevance of Design was Modest.


4. Achievement of Objectives (Efficacy) :

Efficacy is assessed from the three main elements of the project’s objectives: (i) establishing a system for managing biodiversity degradation in the Gulf of Gabes Region; (ii) establishing a functional integrated monitoring system; and (iii) use of participatory approaches.

Establishing a System for Managing biodiversity degradation in the Gulf of Gabes Region

Various sub-strategies were prepared for particular aspects of biodiversity conservation, including for control of evasive species, a manual for managing fishing practices, and managing ships’ ballast. Management Plans were prepared for all six pilot diversity management sites, and, for three sites (the number targeted at appraisal) initial steps were taken to implement the plans. Part of the plans included related infrastructure, although construction was only achieved at one site. Using the international Management Effectiveness Tracking Tool, management efficiency on the three pilot sites increased. Between 2009 to 2012 management effectiveness for Kneiss increased from 37 points to 49 points; for Kerkennah islands, from 20 points to 46 points; and for Boughrara Lagoon, from 16 to 45 points. The results from the pilot sites and sub-strategies were used in preparing a national level biodiversity conservation strategy - hence with utility broader than the project’s stated objectives. Capacity building efforts included 40 training sessions and 1413 trainees as against appraisal targets of 30 sessions and 250 trainees. A Project Management Unit in Tunis and a Project Implementation Unit at Glebes were also created. These various actions have laid the groundwork for better planning and management for biodiversity conservation in the future (project objectives were primarily to establish the monitoring and management basis for conservation rather than actual implementation). The project’s Efficacy at Establishing a System for Managing biodiversity degradation in the Gulf of Gabes Region was Substantial.

Establishing a Functional Integrated Monitoring System

Baseline key indicators were identified for marine fish species, habitats and water quality; trends for marine flora and fauna, exotic and invasive species, and seagrass were monitored; and more detailed inventory/monitoring systems for lagoon and marine species, alien species, and distribution of these species in the Gulf of Gabes, were set in place. A GIS and publically available Information Exchange Center was created with user-friendly access, which integrated project-generated technical, biodiversity and socio-economic data. Thus, while it can be developed further over time to a larger array of data, a Functional and Integrated Monitoring System has been established – Efficacy was Substantial.

Using Participatory Approaches

Methodologies for participation of communities and other stakeholders were prepared and at each pilot site a Local Development Committee was created. Training and awareness campaigns by NGOs helped encourage local knowledge of biodiversity conservation, and encouraged participation in preparation of management plans. However, the extent and depth of participation is not clear, and the ICR (page 6) comments “Community participation focused on capacity building, but did not address the real participation of local communities in decision making processes.” (NB: As mentioned in Section 3b, Task Team Leaders interviewed by IEG considered that the degree of participation was greater than assessed in the ICR) Participatory approaches are reported to have improved in the last year of the project, but this would not have been in time for significant impact.) The project’s Efficacy Using Participatory Approaches was only partially effective and is assessed Modest.


5. Efficiency:

In overall terms the project was cost-effective. (The ICR does not calculate an economic rate of return.) Most physical targets were met, while actual project costs were 21 percent lower than estimated at appraisal.

Another criterion is implementation efficiency. While most of the project’s output targets were met, the implementation period was extended by 2 years, an increase of 50 percent over the original project period. But the inefficiency here was mainly due to exogenous circumstances - a period of social unrest which the ICR estimates held the project back by some 18 months.
Based in particular on the project’s cost-effectiveness, Efficiency is assessed Substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


The project’s objectives were highly relevant to Tunisia’s key need to better conserve its coastal and marine ecology in the face of declining biodiversity and increasing environmental threats - issues which were well recognized in the Country Partnership Strategy. Design captured the technical and capacity-building needs to begin a biodiversity conservation program, but institutional arrangements only partly integrated the activities of the different agencies involved. There were also some shortfalls in the degree of participation preparing the management plans. Nevertheless, in its other two thrusts (better managing biodiversity, and establishing an integrated monitoring system), the project largely achieved its objectives. The project was cost-effective as it implemented most of the intended implementation program at a cost some 20 percent below appraisal expectations. Thus, with the exceptions of weak inter-agency linkages and participatory processes, in most respects the project was relevant, effective, and efficient. Outcome was Moderately Satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The main risk would be whether, without GEF financing and the driving force of a project, a broad interest in biodiversity conservation across influential stakeholder factions (e.g. political, the scientific community, environmental leaders, the general populace, local communities and key actors/lobbyists such as fishermen, municipalities, industries and tourism) would continue to develop. Successful results from the pilot sites, and a continued public awareness program would help consolidate such support. Second, there is a risk that communities might be insufficiently involved in planning and management of bio-diversity sites which may impair organizational sustainability and capability to resist adverse actions from powerful stakeholder interests (e.g. trawlers, developers). And third, a more organically embedded solution is needed than the ad hoc institutional arrangements (the Steering Committee and the two implementation units) created for the project.

Thus, while the program has gained popular support, which is the buttress for reducing such risks, because marine and coastal biodiversity conservation is still a relatively new practice in Tunisia, and needs further embedding in Tunisia’s socio-political mind set, Risk to Development Outcome is still Significant.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:

A good strategic diagnostic was made of biodiversity threats and the improvements in monitoring and implementation capacity needed to counter them. The project’s technical design was sound, based on appropriate specialist skills amongst the Bank team, including in biodiversity management and marine and coastal ecosystems.

The design of the institutional structure for coordinating between agencies was somewhat functional for cross-agency coordination, but it was less suited for fully integrated biodiversity management, and was a creation for the project period and not a long-term solution. The procurement capacities of the implementing agencies were weak, which subsequently contributed to the initial delays in project implementation (though the primary cause was the disruptions caused by the period of social unrest). Overall, though quality at entry was strong in terms of the project’s strategic and technical base, and the Bank brought high level specialist expertise to the preparation, there were a number of shortfalls in the design of the project and its readiness for implementation. Quality at Entry was Moderately Unsatisfactory.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:

Project supervision was thorough, involving bi-annual missions, appropriate specialist skills, and proactive recourse to videoconferencing when missions were not possible during the period of social problems. Most implementation problems were handled practically, and extra effort was placed on assisting the borrower to build capacity in procurement, a critical need. (Each agency had its own procurement methods, and each had to mesh with the Bank’s, Section 11b)

The limited attention at appraisal to the detailed implementation features for grassroots community participation, continued during project implementation. Also, the Bank could have been more effective at promoting integrated biodiversity management between agencies. However, in other respects supervision was hands-on and effective. Thus, overall, the Quality of Supervision was Moderately Satisfactory.

Overall Rating for the Bank’s performance. With quality at entry moderately unsatisfactory, and quality of supervision moderately satisfactory, the overall rating for the Bank’s performance is determined by whether the project’s outcome is rated in the satisfactory range or the unsatisfactory range. Thus, as the project’s Outcome is rated Moderately Satisfactory, the Bank’s Overall Performance is rated Moderately Satisfactory.

Taking the Quality at Entry and the Quality of Supervision together, the Overall Performance of the Bank was Moderately Satisfactory.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government was strongly committed to environmental conservation and improving the management of biodiversity, providing a positive environment for preparing the project and during implementation. Government commitment continued after the social disruptions, helping the resumption of activities. Counterpart funds were provided to full needs and in excess of appraisal expectations. Covenants and agreements were complied with. Coordination of the agencies was effective during preparation of the project, but during implementation, the project Steering Committee met infrequently and was not conducive to integration of activities between agencies. But as in other respects Government supported the project well, Government Performance was Moderately Satisfactory overall.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

For the General Directorate of Environment and Quality of Life, and the International Center for Technology and the Environment of Tunis, responsible respectively for components one (Institutional Strengthening, Strategic Planning and Dissemination), and two (Training and Capacity Building), their activities were largely completed without major problems, and their performances were satisfactory. For the National Institute of Sciences and Marine Technologies, which implemented most of the environmental monitoring and technical studies (component three), some planned activities were not fully completed, mainly because of procurement problems, and performance was moderately satisfactory. For the Coastal Planning and Protection Agency, which handled the Participatory Biodiversity Management Plans at the six pilot sites (component four), the Plans were prepared to reasonable technical standards, and environmental management improvements were noted even during the project period. Experience from the pilots has also informed preparation of the National Biodiversity Strategy. However, a fundamental aim of the plans was that they would be “Participatory. Nevertheless in most other respects the plans were prepared and field activities commenced. performance of the Coastal Planning and Protection Agency was moderately satisfactory.

Nevertheless, the four agencies were able to implement a technologically complex project with many activities, much of them innovatory, and most of which were successful. The Performance of the Implementing Agencies was Moderately Satisfactory.

Taking account of both the performances of Government and the Implementing Agencies, the Overall Performance of the Borrower was Moderately Satisfactory.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The collection and tracking of data relevant to biodiversity conservation, including use of technologically sophisticated measurement techniques, formed a large part of the project’s activities, But the M&E system for the project itself was primarily focused on MIS type data, and was inadequate as concerns monitoring the qualitative aspects of the project. Indicators and measurement processes for assessing project outcomes were limited.

Initially, a combined M&E system covering all activities and agencies was envisaged, but, due to weak performance of the consultant designing the system, it was decided that each of the implementing agencies would use its own data collection and information system to collect data and submit regular reports for compilation by the Project Management Unit. Data was primarily management information (e.g. progress of each activity, numbers of trainees, procurement), and there was not much in the form of evaluative indicators (to assess, for instance, the degree of participation, and stakeholder views on participatory processes in the Biodiversity Conservation Management Plans).

b. M&E Implementation:

The MIS orientation and the independent reporting of each agency continued throughout implementation. Reporting appears to have been regular, and data on project progress and physical achievements was adequate for basic monitoring of project progress.

a. M&E Utilization:

The Monitoring data was a useful tool for managing the project, both for Government and for Bank supervision, although the utility of the data may have been more for the individual managements of each agency than for a broader and integrated vision of biodiversity conservation.

As the biodiversity program continues, an increasing need can be expected for more qualitative information about project performance, and there would be an increasing need for data and analysis to be multi-sectoral and across agencies. The project’s M&E program was primarily restricted to an MIS function. Against the broader evaluation needs, the Performance of M&E was Modest.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The project was classified as environmental Category B, and triggered the Environmental Assessment (OP 4.01) and Involuntary Resettlement (OP 4.12) safeguards. An Environmental Management Plan was prepared, with the project itself as the base for the plan, including emphasis on participation of stakeholders. An involuntary resettlement Process Framework was prepared, primarily to address potential restriction of access to resources and impacts on livelihoods. Social activities under the project included establishment of local development committees at each pilot site, and participation (though insufficiently) of grassroots stakeholders in developing management plans.

b. Fiduciary Compliance:

Financial Management: The ICR advises that the four implementing agencies had acceptable staff capacity, accounting, and internal control systems. Accounting was not computerized, and was done individually by each agency and then consolidated on excel sheets by the Project Management Unit. Audits were done annually and revealed no particular accounting issues.

Procurement: Procurement was difficult throughout implementation, especially at the beginning. Each agency had its own procurement mechanisms that had to mesh with Bank guidelines. Procurement staff, unfamiliar with Bank procedures and often unqualified, required regular and continuous support by the Bank. Procurement delays were common, and two works contracts for a total of $0.9 million had to be cancelled due to insufficient time remaining for implementation.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

The project experience provides the following main lessons, which have been adapted from the ICR:

(i) Biodiversity conservation involving multiple agencies needs effective bridging between the agencies.
Involvement of four agencies was required to implement the project, and there were even more stakeholder factions. Although a project unit and a steering committee were established, the latter seldom met, contributing to virtually separate “projects” under each agency.

(ii) Involvement of the community in biodiversity planning and management is key to establishing ownership and commitment.
The project didn’t do enough to stimulate broad-based community participation, which is likely to have reduced ownership and made the reach for sustained management by communities a more challenging goal.

(iii) In a multi-agency project with institutions unfamiliar with Bank procurement processes, training and repeater training in procurement for all implementing agencies is needed.
None of the four implementing agencies was familiar with Bank procurement processes, and this impaired project implementation. Continuous technical support in procurement processes was provided by the Bank, and there was also a need for repeater training to refresh the skills of existing procurement staff and to train new staff.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is candid and structurally well presented. Particular strengths are its probing diagnostic of issues arising, the learning emerging from the project experience, and the report’s forwards looking orientation as issues are discussed. There are two main areas where discussion could have been expanded. First, it would have been helpful to have had a more focused discussion on how the institutional inter-linkages problem could have been resolved, as replicating the pilots to a larger program depends on a workable institutional framework. Second, the shortcomings in adoption of participatory approaches would have merited a discussion on what the weaknesses were, their impact, and how to resolve the weaknesses. And third, while the ICR needs to preserve an independent outlook, more consultation between the implementation TTLs and the ICR TTL might have reduced the differences in viewpoints on the participation and institutional issues, or been more persuasive. (This observation is not included in the assessment of the quality of the ICR report as it is more a reflection of the broader review process.)
Overall, however, the ICR provides a thoughtful discussion, and its Quality is rated Satisfactory.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions