Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Second National Fadama Development Critical Ecosystem Management Project


  
1. Project Data:   
ICR Review Date Posted:
01/29/2014   
Country:
Nigeria
PROJ ID:
P073686
Appraisal
Actual
Project Name:
Second National Fadama Development Critical Ecosystem Management Project
Project Costs(US $M)
 10.03  10.01
L/C Number:
Loan/Credit (US $M)
   
Sector Board:
Environment
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  04/11/2006
 
 
Closing Date
12/31/2011 12/31/2011
Sector(s):
Irrigation and drainage (50%), General agriculture fishing and forestry sector (24%), Forestry (23%), General public administration sector (3%)
Theme(s):
Land administration and management (29% - P) Water resource management (29% - P) Other rural development (14% - S) Environmental policies and institutions (14% - S) Biodiversity (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Ebru Karamete
Ridley Nelson Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The project was a GEF operation. For implementation it was mainstreamed into the Second National Fadama Development Project (P063622), which was implemented in 18 states. The latter started and finished earlier than the GEF operation being approved by the Board on December 16, 2003 and closed on December 31, 2009. The GEF operation financed incremental costs on sustainable land and water management in 6 of the 18 states participating in the Second National Fadama Development Project.

The objective of the project stated in the Global Environment Facility Trust Fund Agreement (p.16) was: "to enhance the productivity of Fadama areas, and the livelihood systems supported by them, through sustainable land use and water management'. The Project Appraisal Document statement of objectives was similar with the inclusion of six states (p.3) "to enhance the productivity of fadama areas and the livelihood systems they support through sustainable land use and water management in the six participating states".

This Review’s assessment is based upon the formulation of the project objective in the Trust Fund Agreement.

For comparison, the parent project, the Second National Fadama Development Project's PDO as stated in the Credit Agreement (p.24) was: " (a) to increase the incomes of fadama users in a sustainable manner; (b) to empower communities to take charge of their own development agenda; and (c) to reduce conflict between fadama user groups through the financing of investments in human and physical capital, building the capacity of community organizations and government at all levels to deliver services, improving the capacity of the economically-active rural poor to raise their incomes through targeted activities, and promoting socially-harmonious and environmentally-sustainable management of natural resources."

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1. Capacity Building (Appraisal Estimate:US$1.42 million financed by GEF, Actual: US$1.80 million)
The aim of this component was to build the capacity of fadama users and other key stakeholders. While IDA financing (74 percent of the overall component cost including the 'parent' Fadama II project ) supported building the capacity of Fadama User Associations in order to enable them to access project advisory services and to invest in productivity and income enhancement activities, the aim of the GEF financing (6 percent of the component cost) was to enhance the capacity of different stakeholder groups--including relevant federal, state and local government, NGOs, community based organizations, and fadama users in the six targeted states (Imo, Kebbi, Kwara, Kogi, Ogun, and Bauchi) for sustainable land and watershed management.

2. Integrated Ecosystem Management at the Watershed Level (Appraisal Estimate: US$4.08 million, Actual: US$ 3.14 million)
The aim of this component was to improve the management of critical watersheds in pilot Fadama areas to ensure their sustainability and productivity. This objective was to be achieved by implementing technical, social, and location-specific activities with high potential for scaling up and replication. The component sought to: (i) strengthen watershed planning and coordination mechanisms among state agencies; (ii) ensure sustainable management of forest resources by developing community forest reserves in highly degraded and conflict-ridden rainforest and savannah areas in Fadama ecosystems; (iii) develop a lake management plan for a Ramsar site; (iv) conduct a study to understand the impact of upstream reservoir management and river flow regime on Fadama areas; and (v) develop a monitoring plan to improve the management of groundwater and shallow aquifers in selected Fadama areas.

3. Community Sustainable Land Management (Appraisal Estimate: US$3.96 million, Actual: US$3.50 million)
This component aimed to support a range of advisory services, training, information sharing, awareness programs, and adoption of land use practices that will enable fadama users to adopt productivity-enhancing techniques and more profitable marketing, and at the same time ensure the sustainability of the fadama resource base. GEF support was in the form of grant cofinancing, using a demand-driven approach for two types of alternative land use practices: (i)land use changes in critical areas, such as, river banks, flood-prone or ground water recharge areas, and forest or natural habitats of significant biodiversity values; and (ii) sustainable agricultural practices in fadama areas.

4. Project Management and Monitoring and Evaluation (Appraisal Estimate: US$O.57 million, Actual: US$1.57 million)
This component aimed to support project management mechanisms, including M&E plans. While 76 percent of financing for this component was through the IDA Credit, GEF financing (3 percent) aimed to support the full integration of GEF-funded activities into the two main Second National Fadama Development Project subcomponents, while keeping track of the specific inputs, outputs, and impacts of the GEF activities.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs

Total project costs estimated at the time of appraisal was US$11.45 million, the actual total project cost was not presented in the ICR.

Financing:
Project funding was US$10.03 million at appraisal and the disbursements at project closing was US$10.01 million. Local communities were estimated to contribute US$ 0.44 million. Actual figures for local communities at project closing was not presented in the ICR. The project team subsequently stated that the actual figure for local communities' contribution was US$ 0.34 million at project closing.

Borrower Contribution:
At the time of appraisal, the Borrower was estimated to contribute US$ 0.98 million. Actual figure at project closing was not presented in the ICR. The project team subsequently stated that the actual borrower contribution was US$ 0.96 million at project closing.

Dates:
The project closed at the original closing date of December 12, 2011 without any extensions. A level 2 restructuring on August 16, 2010 reallocated funds among project categories in order to complete activities by project closing.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial
At the time of project appraisal, about 60 % of Nigeria’s rural population (about 75 million) lived in extreme poverty (ICR p. 1). Natural resources served as the main source of livelihoods for most rural people, but productivity was low and had not improved. At the same time, Nigeria’s natural resources suffered from over-exploitation and unsustainable land use practices. Weak management of protected areas, posed a serious threat to the maintenance of ecosystems and habitats as well as rural livelihoods. The natural resource sector was also facing deforestation, large-scale land clearing, and flood-plain encroachment, mainly due to efforts to expand agricultural production. This led to severe land erosion. The loss of valuable topsoil, significant siltation of water sources, and flooding affected the sustainability of the Niger and Benue River systems, including the Fadama areas, which are floodplains and low-lying areas overlaying shallow aquifers that play an important role in recharging groundwater and supporting productive vegetation. Soil losses reduced the productivity of the agricultural resource base for rural communities, and frequent floods destroyed fields and homes. The main constraints to sustainable fadama land management included: (i) the lack of adequate land use and land management plans that prevents the conversion and open access to fadama resources; (ii) lack of awareness of local communities, and weak capacity of extension advisors on sustainability considerations; (iii) lack of adequate coordination and integration of Natural Resource Management policy and strategy at all levels of government; (iv) limited capacity in related institutions.

To address these and other issues, the Federal Government of Nigeria developed a medium-term plan aimed mainly at growth of the non-oil sector. The plan was implemented through a National Economic Empowerment and Development Strategy over 2004–09 and environment was one of the four key dimensions of this strategy, which envisioned a large, strong, diversified, sustainable, and competitive economy for Nigeria by 2020. The project development objective was fully relevant to this strategy.

The objectives were also relevant to the current Country Partnership strategy (2010-2013), specifically with Pillar I: Non-oil sector growth, and areas of agricultural productivity and environmental and climate change management.

b. Relevance of Design:
Modest

There was a lack of a causal chain and measurements in the design to support attribution of the project to achieving gains along the results chain, beginning with sustainable land and water management techniques and ending with the objective of increased productivity. Project activities included: building capacity of stakeholders in sustainable land and watershed management, strengthening watershed management and coordination, developing community forest reserves, developing lake management plans, developing a monitoring plan for groundwater resources and providing finance for demand driven alternative land use practices. However, the main objective was to increase productivity in Fadama areas and from the design it was not clear how these means would actually lead to the productivity increase. Therefore, selection of "productivity" as project development objective, rather than for example "sustainable land and water management capacity increase" was a key shortcoming, Furthermore, the scope of project in relation to funds was too ambitious. Project funding of US$10 million was very limited to cover what was proposed. As can be seen in Section 4, project activities could cover only a small part of Fadama Areas. Either the coverage and limitations of the project should have been made clear in the objective or the design and funding should have matched the ambitious objective. We treat this as a design issue here for rating purposes but it relates to the disconnect between objectives and design.

Finally, the key performance indicators to measure the development objective were not well formulated, they measured sustainable water and land use management achievements but not the main objective of enhancing the productivity of Fadama areas and the livelihood systems they support. Also, project design could have included outcome indicators, such as soil fertility, soil health, and vegetational changes in order to measure land and water management impacts that, in turn, impact on longer term productivity.


4. Achievement of Objectives (Efficacy) :

The achievement of the objective of the project, to enhance the productivity of Fadama areas and the livelihood systems supported by them through sustainable land use and water management', is rated modest.


The evidence is drawn from M&E data, a Beneficiary Impact Assessment carried out at the end of the Project, and 2008 and 2010 Agricultural Production Surveys carried out by the Government. The Beneficiary Impact Assessment was done in six project regions with a sample of 180 beneficiaries and with control group comparisons.

Outputs and Intermediate Outcomes:

Sustainable Land Use

  • 100 % of the Fadama Community Associations adopted sustainable land use planning practices in the implementation of their local development plans. However, it was not clear how many associations were actually covered (number of Fadama Community Associations under the parent project is 1,470).
  • Members of the State Watershed Subcommittees in 6 states received training on sustainable land use planning at watershed level, development planning at community level, integrated farming and sustainable agriculture in Benin.
  • The project identified and supported management of 18,800 hectares of forest reserves.
  • Combined area under sustainable land management in the Andiwa Lake, Lake Oguta and Eriti Watershed rose from baseline of 7,421 hectares to 11,635 hectares by the end of the project.
  • Sustainable land management improved under 11,635 hectares (87 % increase from the baseline of 7,421 ha).
  • Sustainable Land Management practices were adopted by at least 35 % of Fadama Community Associations in six intervention states (achieved 100 % of the target) also, Sustainable Land Management practices increased by 80 % in six sites (exceeded the target of 3 pilot sites).
  • 489 Fadama User Groups are implementing alternative livelihood activities.
  • 10 different alternative livelihood activities (e.g. apiary, grass cutter and rabbit rearing, fattening of ruminants, snailry), were funded by the project, which represented 438 sub-projects.

Sustainable Water Use
  • The project strengthened the capacity of stakeholders at the institutional and watershed level by equipping them with skills to carry out watershed management framework.
  • Federal level agencies and ministries benefited from various training, workshop and study tours.
  • At project inception no watershed management framework was in place for Fadama Community Associations to adopt and by carrying out four studies, developing training toolkits, developing farming system models, a framework was established. Fadama Community Associations can now prepare and implement their frameworks for watershed management.
  • 47 Local Development Plans were produced and implemented. However, this was small compared to the 1,247 Local Development Plans supported under the parent project.
  • Review of watershed policies and regulations were carried out by the project and recommendations were implemented effectively.
  • The project established State Watershed Subcommittees in 6 participating states and facilitated 98 quarterly meetings for State Watershed Subcommittees with representatives from 12 sectors.
  • The project financed a range of advisory services, including training, capacity building and awareness campaigns on watershed management.
  • A study on improved groundwater management and impact on reservoir management was completed.
  • Watershed management coordination capacity was built in 60 % of the six participating states (achieved 100% of the target).
Outcome:

As mentioned in Section 2a, this project financed incremental costs on sustainable land and water management in 6 of the 18 states participating in the Second National Fadama Development Project. Hence, in the areas the project was implemented, the beneficiaries were the same as the 'parent' Fadama II project but the activities financed were different. The 'parent' project financed sub-projects related mainly to poverty reduction and the GEF project covered sub-projects on sustainable land and water management although, in practice the boundary between the poverty reduction activities and the sustainable resource management activities appears not to have been very clear cut.

As mentioned in Section 3.b, the key outcome indicators measuring the productivity increase of Fadama areas and supporting livelihood systems, are lacking. The ICR provided some data to show that productivity of Fadama areas and the livelihood systems they support increased during the project period based on the adoption of sustainable land management practices. However, the quality of evidence on productivity increase is questionable.

The ICR reported that (p. 12): "The productivity of the high value crops grown by the beneficiaries were tracked during the life of the project and accordingly the yield of 18 cultivated crops in the intervention sites increased significantly during the period, ranging from 0.61 % to 275.52 %." However, it was not clear if the project actually monitored beneficiaries' productivity. The detailed table that the ICR provided (p. 30, Table 2.6) was from 2008 and 2010 Agricultural Production Survey (of the Government) covering the six GEF funded states and showed the productivity of major crops in Project intervention areas, It is not clear if the surveys included project beneficiaries with adequate control groups. Also, showing only two years of data, a very short span, without reference to rainfall pattern raises additional concerns about the validity of these incremental gains and the attribution to the project. Data from two of the states uses baselines that were reported as state averages making comparison questionable.

The ICR reported that (p. 12), 43,568 ha of land was put under sustainable land management in six intervention sites addressing land degradation, desert encroachment and carbon sequestration. The project helped reduce soil erosion and prevent bush fires. 63 % of beneficiaries confirmed soil erosion reduction because of the project (however, it appears that no direct soil loss measurements were carried out). Annual bush burning reduction was reported to have been 40-60%. Several examples of livelihood improvements were given in the ICR although the extent to which these were wholly attributable to the GEF incremental investments as compared to the parent project together with the GEF-supported incremental investments is not clear:
  • Over 60 participating communities implemented alternative livelihood activities. The beneficiaries implemented about 10 different alternative livelihood activities (e.g. apiary, grass cutter and rabbit rearing, fattening of ruminants, snailry), which represented 438 sub-projects. These activities generated Nigerian Naira 11.5 million as revenue for beneficiaries (approximately US$ 70,000) (ICR p. 15).
  • The beneficiary survey conducted at project closing showed that the project had an impact on the change in the level of expenditure of the 7,688 household beneficiaries when compared to the non beneficiaries with a net impact of N 8, 667 (approximately US$ 54) (ICR p.40). However, it is not clear how to interpret this as a project attributable gain, in particular whether it can be attributed to a commensurate increase in income, whether it came from a productivity gain, whether it was household or farm investment expenditure, how much was related to a project supported investment, and how comparable were the beneficiary and non-beneficiary samples with respect to initial expenditure status.
  • Through the capacity building intervention, over 50 % of the project beneficiaries are reported to be now empowered, and willing to continue implementing sub-projects in community afforestation, rearing of grasscutters and snails, and woodlot and orchard cultivation.
  • Trees planted in project sites via community woodlots, shelter belts, road side tree planting, orchards and wind breaks are reported to have reduced water and wind erosion, served as a carbon sink, stabilized soils, helped stabilize riverbanks, enhanced the river filtering system, and performed other ecological functions that reduced and reversed land degradation and improved the productivity of Fadama areas. Most of the trees planted were economic trees, which generated economic returns and were therefore more likely to be maintained. However, it is not clear what the evidence was for these claims of erosion reduction and other benefits at the project sites.

Due to issues with the reliability of evidence on enhanced productivity and livelihoods in the six project state areas, the achievement of the project objective is rated modest.

5. Efficiency:

Modest.
During the appraisal an economic rate of return was not calculated on the grounds that the investments made under the project were demand-driven and, therefore projecting how resources would be allocated was not possible. Although an economic rate of return was included in the ICR using an analysis undertaken for the IDA funded Second National Fadama Development Project, the methodology was incorrect. (The analysis took a 40 % ERR for the IDA investment in Second National Fadama Development Project, which included two additional components for infrastructure investment and pilot asset acquisition support related to the GEF project and incorrectly assumed that the rate of return would be roughly proportional to the relative investment shares, giving an ERR for the GEF project of 18 percent. A 4 % additional return from the net benefits associated with the GEF incremental costs was added.)

In any case, there were issues in the ERR calculation with coverage and sampling. It was partial, including only the alternative livelihood activities on the grounds that activities such as capacity building and project management did not produce direct economic benefits. Four types of livelihood projects were selected that constituted 37 % of the sub-projects (the total number of sub-projects was 958), thus only a small proportion of the total project costs. The coverage of the economic analysis was roughly 13 % of project costs (35 % for Component 3 and within Component 3, 37 % for the 4 types of sub-projects). However, the ICR (p. 35) reported that the types of sub-project were chosen based on the high level of interest exhibited by beneficiaries in continuing with their production after the end of the project. Thus it was not a random or representative sample. The interest criteria was likely to have selected for the better sub-projects. The sub-projects chosen were: apiaries for honey production, teak woodlots for pole production, establishment of a grasscutter farm with a start-up size of two families, and rearing of 30 male and 10 female Belami sheep. A 16 % discount rate was used for the analysis. Results of the analysis indicated that economic benefits of these sub-projects were high (economic rate of returns ranged from 37-78 %). The ICR reported a total ERR of 212 %, it appears by erroneously adding up the 4 sub-project type ERRs.

Finally, it is not clear why, rather than making assumptions about the benefits and costs in the sub-projects, actual data from sampled sub-projects was not used, particularly for sheep and honey which would usually give measurable benefits quite quickly. As noted above, the ICR reports that (p. 14 and 15) the livelihood activities generated approximately US$ 70,221 in revenues over 7,668 beneficiary groups. However, comparing this benefit (US$70,221) with the cost of Component 3 (US$ 3.5 million) which financed these sub-projects, the revenues appear low, raising additional concerns about the validity of the ERR assumptions .

Looking at cost effectiveness in terms of operational or administrative efficiency, the project encountered some administrative inefficiencies early in the project. There was a 32-month lag in disbursements as of September 2009, and the Panel for Quality Assessment of Lending Portfolio (FY09) rated implementation progress as moderately satisfactory due to problems in financial management and procurement and it was recommended that subsequent supervision missions should address the weaknesses in the fiduciary system (for details see Section 11. b). These issues were addressed around mid-term and the project disbursement accelerated but it was a very slow start.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of objectives is rated high due to close linkages of the project with country priorities, relevance of design is rated modest due to lack of causal chain in the results framework and lack of key performance indicators measuring the productivity enhancement objective; efficacy is rated modest due to limited evidence on the achievement of the objectives on productivity and livelihoods. Efficiency is rated modest also due to limited evidence and the difficulty of drawing from the ICR efficiency analysis due to errors in methodology.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

This review largely agrees with ICR's risk to development outcome (ICR p. 18-19) ratings with the exception of institutional risk.

Institutional risk. The project supported capacity building and efforts to create awareness at the community level particularly Fadama User Groups through training and various capacity-building interventions. The ICR notes that the project also established a sense of ownership among beneficiaries and empowered them to take control of their own affairs. Training was provided to the Federal and State level agencies to strengthen their skills in sustainable land-use planning at the watershed level, monitoring and evaluation, watershed policies and regulations reviews, and development and synchronization of local development plans. However, it is important to note that very few local development plans included and implemented sustainable land and water management techniques (only 47 out of 1,247) and it was not clear how many Fadama Community Associations actually became aware of these techniques. Even if sustainable land and water management techniques were included and implemented through Local Development Plans, there are concerns about the sustainability of local development plans in general.

Economic and financial risk. The risks identified at appraisal such as funds diversion and misuse, collusion with service providers and cost escalation, were mitigated by the project team by making use of the financial mechanisms and expertise developed in the Second National Fadama Development Project. The project provided community level training in grant management, which improved beneficiaries’ understanding of bookkeeping and increased their capacity to generate revenue. However, there is evidence that some types of sub-project are more likely to be sustained by farmers than others. Operation and Maintenance is being addressed by the setting aside of 5% of sub-project costs. There is also provision of full public disclosure of expenditure by associations. The fact that there is a third umbrella Fadama III Project following on gives some expectation of consolidation and sustainability.

Environmental risk. No safeguard issues surfaced in relation to involuntary resettlement and land acquisition problems in the intervention areas. Most project activities were environmentally friendly.

Social risk. The project is reported to have enhanced the livelihoods and living standards of beneficiaries although the evidence is limited. The project’s development outcomes should also be sustained through the community level institutions created, that will most likely continue to support its members, i.e. Fadama Community Associations and Fadama User Groups and through the government’s commitment to continue co-financing key activities through the follow up Fadama III Project. The risk of farmers going back to their original practices appears low due to the enhanced income potential of the sub-projects but the evidence suggests that this will be variable by type of activity therefore it can be expected that some activities will replace others of less interest.

On balance, due to institutional risks and some uncertainty about the sustainability of the sustainable land and water management planning and implementation activities and uncertainty about the productivity incentives to sustain interest in longer term resource management, Risk is rated moderate.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
As mentioned earlier, the Project was mainstreamed into with the Second National Fadama Development Project Initially, it was intended to be fully blended and implemented simultaneously, however, due to GEF processing delays, the project was approved two years after the Second National Fadama Development Project. The project areas of intervention and beneficiaries were the same (six states) as those of the Second National Fadama Development Project and the same institutional arrangements were used for the two projects. However, the Global Environment Trust Fund Agreement did not specifically state the six states for the project area.

Project design, especially arrangements for fiduciary processes and M&E, benefited from implementation arrangements for the Second National Fadama Development Project. However, as addressed earlier under Relevance of Design, the team failed to put in place proper outcome indicators within the results framework that adequately measured the PDO. Moreover, there was a disconnect between the ambitious objectives and the means provided to achieve them.

The institutional design was sound. It tried to sharpen the focus of the GEF interventions; it included a Fadama GEF Desk Office, headed by a natural resource officer from the Federal Ministry of Environment, to oversee and supervise the implementation of GEF interventions at the national level. Also, the institutional set up included: subcommittees to coordinate watershed management at the state level; and established the linkages between the project and Second National Fadama Development Project through (i) the adoption of the same beneficiaries in the intervention areas, the use of the same local development plans to express the needs of the communities; and, (iii) the use of the same local facilitators, and implementation agency.

The preparation team conducted an analysis of Fadama areas at appraisal and incorporated lessons from the National Fadama Development Project (Fadama I), the Local Empowerment and Environment Management Project, and successful interventions under the main Second National Fadama Development Project. The key lessons reported and adopted were to (ICR p. 6): (i) empower communities with resources to improve their capacity to implement subprojects; (ii) adopt a socially inclusive approach to local development plans, which were considered very effective in managing community conflicts; and (iii) harmonize local development plans to standardize subproject documents and technical designs.

Given that stakeholder consultation and involvement are critical to the success of community-driven development projects, the team consistently ensured the involvement of all stakeholders, including the beneficiary states, at each stage of project preparation. At the Federal level, the team ensured strong collaboration with the relevant ministries. Key development partners, particularly the African Development Bank, also participated fully.

The appraisal efforts included an assessment of the main risk and mitigation needs. Risks included concerns that Fadama users would be unwilling to practice new, sustainable approaches to land use; insufficient capacity at the state level; and the government's unwillingness to create coordinating mechanisms for integrated watershed management.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:
The evidence provided by the ICR (p. 20), on quality of supervision merits a satisfactory rating. However, this review's rating is moderately satisfactory due to the fact that the team did not deal with the problem of lack of adequate outcome indicators.


According to the ICR (p. 20), the Bank built a team with diverse expertise. 8 joint supervision missions were undertaken, including the Mid-term Review. During the missions, the Bank team worked closely with project staff to address key implementation issues and make recommendations to resolve them. The Bank also maintained working relations with the project staff and was effective in reviewing and clearing documents, including providing no-objections from the project staff. The Bank organized a quality assessment review conducted by the Quality Assessment Group in FY09. The review was conducted based on an implementation progress rating of moderately satisfactory, due to initial fiduciary problems (in financial management and procurement) and M&E problems. According to the ICR, the quality assessment found that the Bank provided good advice and solutions to problems but recommended that subsequent supervision missions should address the weaknesses in the fiduciary system. The Bank team and project staff implemented the recommendations of the panel, and implementation performance improved, after the Mid-term Review. However, there is no information in the ICR if the team ever attempted to include new outcome indicators to measure the development objective more adequately. This was weakness that could have been corrected. But, as a result, the M&E system failed to monitor adequately the productivity enhancement objective.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
According to the ICR (p. 20) the Government showed a high level of commitment: It facilitated the preparation process, put in place dedicated officers to the project management unit, and met all of the effectiveness conditions. Although there were initial challenges in the release of counterpart funds, these difficulties were resolved eventually, and the government made regular and timely payments of counterpart funds. There was synergy between the Federal Ministry of Agriculture and Rural Development and the Federal Ministry of Environment which positively affected implementation. The Federal Ministry of Environment assigned a highly qualified staff to manage the Fadama GEF desk office.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:
The project utilized the implementation structures and Project Management Unit of the Second National Fadama Development Project integrated in the Federal Ministry of Agriculture and Rural Development. The Project Management Unit served as the lead project implementation agency, facilitated all preparatory missions from identification to appraisal, and worked in harmony with the Bank team to ensure the quality of key project documents. The Project Management Unit ensured smooth implementation of day-to-day activities, discussing and resolving key implementation issues with the Bank teams. The Project Management Unit complied with the Bank’s reporting procedures, especially in the areas of procurement and financial management.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
As noted in section 3.b, key performance indicators to measure the development objective were not adequately set. Indicators set measured sustainable water and land use management activities at an output level but not the main objective of enhancing productivity of Fadama areas and livelihood systems they supported.


The project used the M&E framework designed for the Second National Fadama Development Project At the national level, a monitoring template that contained the output indicators was used to track data on a quarterly basis. At the state level, the M&E officers of Second National Fadama Development Project were responsible for monitoring and
reporting data. The M&E system also included Impact Assessment Survey done at the end of the project and utilized Agricultural Production Surveys of the government.

b. M&E Implementation:
A major M&E problem occurred prior to the Mid-term Review, as qualitative updates of ongoing activities were used for reporting against quantitative indicators. This resulted in a moderately satisfactory rating for M&E in the September 2009 ISR. The national M&E unit resolved the problem by organizing training for state M&E staff, and the January 2010 ISR rated M&E as satisfactory.


Also during the mid -term review another issue came up: some state M&E officers were focusing more on the Second National Fadama Development Project than on GEF interventions, which inhibited data collection and reporting on those interventions. The national M&E unit trained the state M&E officers, and developed a standardized monitoring format to be used. According to the ICR, this action improved the collection, reporting, and use of project data.

a. M&E Utilization:
No information was presented in the ICR on M&E utilization.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The project was classified as Category “B” and complied with all safeguard requirements. The project triggered two safeguard policies: Environmental Assessment (OP 4.01) and Involuntary Resettlement (OP 4.12). Under the main Second National Fadama Development Project, an Environmental and Social Management Framework, including environmental and social checklists, was prepared and adopted for the project. The checklist was used to screen the 958 sub-projects across the six intervention sites. No safeguard issues were identified during implementation. The project developed, reviewed, and certified forest management plans in accordance with Bank standards for forest management. These plans were developed with full participation of locally affected communities, consistent with the principles and criteria of responsible forest management. The project team consulted with community leaders and documented land acquisitions to forestall future conflicts. The ICR did not report safeguard compliance ratings but the the task team stated that the safeguard ratings were satisfactory.

b. Fiduciary Compliance:
According to the ICR (p. 9), the project complied with the Bank’s fiduciary policy and procedural requirements. Initially, an assessment review conducted in FY 09 by the Quality Assessment Group, gave an implementation progress rating of moderately satisfactory mainly due to fiduciary problems. Accordingly, project implementation was slowed because states did not withdraw funds in a timely way from the imprest accounts into which funds were transferred. At mid-term, a decision was made to stop replenishing accounts of the states if they had not submitted retirements of previous accounts acceptable to the project desk office within 21 days following the end of the quarter. This measure corrected the situation and the project did not experience disbursement delays thereafter.

Also, during the mid-term review it became clear that the accountant’s significant workload prevented him from dealing with critical issues. The Mid-term Review recommended the appointment of an accounting assistant and employment of additional staff improved the accounting and reporting of project funds and disbursement. The Project Coordination Unit built capacity at the local level and improved the overall financial management system, including computerization of the accounting system. The project initially encountered problems with the payment of counterpart funds. Following intensive consultation with government, regular payments were subsequently received. The financial management team of Bank missions after Mid-term Review rated the financial management system as satisfactory. The project submitted regular Interim Financial Reports, and accounts were fully audited and up to date. A direct transfer mechanism was established, which allowed funds to be transferred directly to beneficiaries upon satisfactory completion of their respective Local Development Plans.

Procurement was judged satisfactory by the Bank’s procurement team, and no major procurement issues arose during implementation, except that the Project Coordination Unit staff of the 'umbrella' Project could not cope with the volume of procurement. The problem was resolved by hiring more staff, but some studies were delayed initially by procurement problems. Most procurement was done at the federal level. At the state level, the project adopted a community-based procurement approach for sub-project activities.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Unsatisfactory
The relevance of design is rated modest due to lack of causal chain in the results framework and lack of outcome indicators measuring the PDO, efficacy is rated modest due to weakness of the evidence on the achievement of outcomes. Efficiency is also rated modest due to lack of evidence and issues with the efficiency analysis.  
Risk to Development Outcome:
Negligible to Low
Moderate
Institutional risk is evaluated higher than the ICR's risk rating due to the fact that very limited number of local development plans included sustainable land and water management techniques. There are also some questions about the incentives for sustaining longer term resource management activities. 
Bank Performance:
Satisfactory
Moderately Satisfactory
The rating differs mainly because the Bank team did not perform well in ensuring outcome indicators that actually measured the PDO and in ensuring consistency between the objectives and the design.. 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Unsatisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The first lesson is formulated by IEG based on project experience in the design of indicators. The remaining three, with some adaptation in the fourth lesson, are from the ICR (p.21/22).
  • In order to better monitor and measure project outcomes, outcome indicators reflecting the objectives and the links to the results chain should be designed at the outset.
  • Community participation, ownership, and empowerment are key to the success of community-driven development projects. The Project showed that the direct transfer of funds to the communities (Fadama Community Associations and Fadama User Groups), as well as the capacity-building programs organized for them, improves their ability to prepare and implement projects and builds a sense of ownership and empowerment. This leads to better local monitoring, data collection, and coordination.
  • Sustainable livelihood interventions, with significant advocacy and awareness programs, have proven effective in ensuring environmental sustainability. Natural resource management interventions have more impact if they are linked to income-generating activities that improve the living standards of beneficiaries.
  • Blending separate but overlapping development projects requires a thorough assessment of, and measures to dovetail, the existing implementation capacity and arrangements. As noted, the project was integrated into the Second National Fadama Development Project and used its fiduciary and M&E systems, but the staff could not cope with the additional workload. Financial management, procurement, and M&E activities were delayed. Future projects should incorporate specific measures and actions to mitigate problems that would arise from such arrangements.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

While there is some useful information, the ICR has a number of significant shortcomings. The quality of evidence presented to assess project outcome is insufficient. There is a lack of data on project financing by different sources. Full reporting on safeguard compliance is missing. The economic analysis calculations to measure project efficiency are incorrect and there is little evidence on the cost effectiveness of the use of project resources or operational efficiency. The ICR also did not present sufficient information on M&E.

a. Quality of ICR Rating: Unsatisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions