Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Community-based Rural Development


  
1. Project Data:   
ICR Review Date Posted:
04/21/2014   
Country:
Ghana
PROJ ID:
P081482
Appraisal
Actual
Project Name:
Community-based Rural Development
Project Costs(US $M)
 73.94  100.55
L/C Number:
C3964, C4485
Loan/Credit (US $M)
 60.00  83.79
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
 12.00  13.67
Cofinanciers:
Agence Francaise de Développement
Board Approval Date
  07/29/2004
 
 
Closing Date
02/15/2009 06/30/2011
Sector(s):
Irrigation and drainage (20%), Health (20%), Water supply (20%), Roads and highways (20%), Agro-industry (20%)
Theme(s):
Rural services and infrastructure (33% - P) Rural non-farm income generation (17% - S) Participation and civic engagement (17% - S) Micro Small and Medium Enterprise support (17% - S) Rural markets (16% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Elena Bardasi
Robert Mark Lacey Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
According to the Project Appraisal Document (PAD, page 5): "the objective of the project was to strengthen the capacity of rural communities to enhance their quality of life by improving their productive assets, rural infrastructure and access to key support services from private and public sources."

According to the Development Credit Agreement (Schedule 2, page 14), “the objectives of the Project are to strengthen the capacity of rural populations to enhance the quality of life of beneficiary rural communities through: (a) building and strengthening the capacities of national, regional, district, area councils and rural communities to deliver services and manage resources; (b) transferring technical and financial resources for the development, management and maintenance of infrastructure and income generating activities for the economic and social well-being of beneficiaries; and (c) promoting the convergence of decentralization efforts in accordance with the National Decentralization Action Plan."

Additional Financing of US$ 22million equivalent was approved in June 2008, in order to reconstruct public goods in flood-affected northern Ghana, support more decentralized provision of services by funding local government authorities, and scale up capacity building in connection with expanding public works. The project objectives remained unchanged.

The Project Paper on the proposed Additional Financing (dated June 2, 2008), reports (page 6) that a new Financing Agreement would amend, restate and supersede the original Development Credit Agreement. Inter alia, the new legal document would eliminate inconsistencies between the statements of objectives in the PAD and the original DCA by adopting the PAD’s statement, on which “Government and task teams have based analysis, results framework and performance measures” (Project Paper footnote 6, page 7).

This Review is based on the statement of development objectives originally contained in the PAD and confirmed in the amended legal agreement following the approval of the Additional Financing.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project had six components:

Component 1: Institutional Strengthening and Capacity Building (appraisal: US$8.12 million; at Additional Financing (AF): US$12.16 million; actual: US$9.21 million): This component sought to support the government decentralization agenda by strengthening the capacities of local government institutions, more specifically: (i) strengthen the capacity and skills of the Ministry of Local Government and Rural Development (MLGRD), the 10 Regional Planning Coordinating Units (RPCU), 138 District Assemblies (DAs), Area Councils (ACs) including traditional authorities within the ACs , rural communities, Non-Governmental Organizations (NGOs), Community-Based Organizations (CBOs), private and public service providers; (ii) strengthen the financial and procurement management capacities of MLGRD and regional and district tender boards, committees, councils, ACs and other procurement entities; and (iii) strengthen the decentralization Secretariat to pursue the decentralization agenda pursuant to the Local Government Service Act. The component was also meant to provide training, public communication programs, technical support and provision of logistical support to enhance communities' participation in decision making processes, and the performance of various levels of government in their statutory roles in planning and managing community driven initiatives.

Component 2: Infrastructure for Agricultural Development (appraisal: US$20.16 million; at AF: US$32 .00 million; actual: US$21.08 million). The objective of this component was to provide facilities and resources to improve farm and nonfarm activities and generally enhance socioeconomic activity in rural areas and the incomes of beneficiaries. More specifically, it was meant to provide grants or advances to beneficiaries for: (i) water for agriculture, including small-scale irrigation from stream-flow diversions, pumping and run-off management, ponds, dugouts and tube wells; earthen banks, bunds and other flow controlling structures; (ii) improving skills for management of facilities, including development of basic skills of beneficiaries for the appropriate application of recommended best catchment management practices and operation and maintenance of investments in water; (iii) rehabilitation of priority feeder roads, provision of culverts and appropriate drainage; and upgrading of farm-to-village tracks and trails to facilitate use of bicycle trailers, donkey carts and other light vehicles; and (iv) post-harvest infrastructure, including provision of storage facilities, drying floors, cribs, crop processing and smoking units, and other processing equipment and facilities. The component was meant to finance consultant services for engineering design, contract supervision and training wherever necessary. Maintenance of facilities was assigned to the responsibility of the beneficiary communities. The project aimed to train the beneficiary communities and provide them the skills for doing so. The PAD indicated that until the responsibility for the maintenance of feeder roads was transferred to the districts, the Department of Feeder Roads (DFR) was required to include any rehabilitated road in their annual maintenance plans and a Memorandum of Understanding had to be signed between the MLGRD and the DFR to that effect.

Component 3: Rural Enterprise Development and Learning Centers (appraisal: US$7.90 million, at AF: US$8.37 million; actual: US$5.9 million). This component was intended to develop skills and create employment in rural communities by assisting rural enterprises to properly manage and provide ready market for agricultural produce for processing for value addition and increased shelf life. More specifically, it was meant to provide grants or advances to beneficiaries for: (i) training in technical and business management for the acquisition of employable skills in rural areas, establishment of agro-based micro enterprises, and upgrading and rehabilitation of equipment; (ii) establishment of learning centers to serve as focal points for processing units, marketing outlets, and empowering the poor through the acquisition of knowledge and skills; and (iii) establishment of rural enterprises, including co-operative business activities, market research, business linkages and joint ventures. The PAD specified that a learning center was to be organized around a prospering small private enterprise that accepted for training people willing to develop similar business. Trainees who acquired skills from the center were to become part of the learning and production network and could continue to be associated with the center or encouraged to set-up their own business. The principal target groups for this component was identified in existing farm and non-farm entrepreneurs, under employed and unemployed rural population (made up of rural youth, rural women and graduate apprentices).

Component 4: Infrastructure for Social and Human Development (appraisal: US$21.97 million; at AF: US$38.84 million; actual: US$47.44 million). This component was designed to provide an enabling environment for learning and for delivering health services. It was meant to support the building or renovation of school facilities as well as community health and nutrition centers. It also aimed to provide potable water in areas subject to waterborne diseases to relieve the disease burden, enhance the productivity of the rural population, and improve women’s and children’s access to clean water. More specifically, this component was meant to finance: (i) the rehabilitation and improvement of existing rural community educational structures and the construction of new community health compounds and nutrition centers; (ii) the construction of health compounds for the Community Health Planning Services; (iii) the rehabilitation of nutrition centers in association with the World Food Program; and (iv) the provision of potable water and sanitation facilities for rural communities.

Component 5: Community-Based Natural Resource Management (appraisal: US$4.15 million; at AF: US$5.1 million; actual: US$3.07 million). This component was meant to strengthen the capacity of environmental and sanitation subcommittees, Area Councils and community level organizations to enhance environmental governance and integrated management of land and water resources. Specifically, this component aimed to support: (i) the preparation of natural resources management plans in District Assemblies, Area Councils, and local communities in collaboration with relevant public agencies including the Environmental Protection Agency, Savanna Resources Management Center, Resource Management Support Center, Ministry of Food and Agriculture, Ministry of Lands and Forestry and the Wildlife Division; (ii) the establishment of tree and tree-crop nurseries for the production of seedlings, adoption of soil and water conservation measures, control of bush fires, and establishment of wood lots; (iii) the rehabilitation of critically degraded watershed areas to ensure regular flow of water and effective management and use of water resources; and (iv) the provision of incentive schemes to natural resource collectors, processors and traders, wood carvers, traditional medical practitioners, wildlife traders and breeders and rural eco-tourism operators, excluding loggers, to increase income earning capacities of beneficiaries and provide skills to rural unemployed.

Component 6: Project Management (appraisal: US$6.89 million: at AF: US$10.26 million; actual: US$13.85 million). The project aimed to provide technical and financial support to the Ministry of Local Government and Rural Development, and other implementing agencies for coordination and day-to-day management and monitoring and evaluation. Specifically, this component was meant to provide technical assistance and services to communities to prepare development and investment plans, feasibility studies, detailed engineering designs, procurement, supervision of contracts and monitoring. Similar packages were meant to be provided at the national and regional levels to guide and coordinate project implementation, provision of credit, monitoring and evaluation.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs
The total estimated project cost at appraisal was US$73.94 million, including physical and price contingencies of US$4.75 million. The project paper proposing the Additional Financing estimated total project cost at US$106.73 million. Actual cost was US$ 100.55million, 36% higher than the appraisal estimate and some 6% less than the revised estimate at the time of the Additional Financing. It is noteworthy that project management costs were double the original appraisal estimate and exceeded the revised estimate at the time of the Additional Financing by 35%. The ICR (page 7) comments that the project incurred higher than expected consulting and operating costs, which were covered by reallocating savings from civil works and initially unallocated funds.
Financing
The originally approved IDA Credit was for SDR40.9 million (US$60 million equivalent). Additional financing of US$ 22 million brought the total anticipated IDA contribution to US$82 million. Due to appreciation of the SDR against the US dollar, the actual dollar equivalent of the IDA contribution at closure was US$83.79 million. According to the disbursement profile in the ICR Data Sheet, both the original Credit and the Additional Financing were fully disbursed.
The project received US$13.67 million equivalent of co-financing from the French Development Agency ( Agence Française de Développement – AFD). This was US$1.67 million higher than the appraisal estimate of US$12 million due to the appreciation of the euro against the US dollar.
There were also to be contributions of US$2.56 million from District Assemblies and US$2.40 million from local beneficiaries. However, the Assemblies made no contribution and that of the local beneficiaries was US$1.68 million, 30% less than the appraisal estimate.
Borrower Contribution
The project benefited from a contribution from the Government of Ghana of US$1.41 million, considerably less than the US$8.9 million planned at appraisal (the project team clarified that this was due to changing macroeconomic conditions which affected the ability of the Government to contribute).
Dates
The original closing date of February 15, 2009 was extended to June 30, 2010 when the Additional Financing was approved. The Government requested and obtained a further one-year extension of the closing date from June 30, 2010 to June 30, 2011. This extension was intended to complete activities related to: (a) rehabilitating and making spot improvements on 71 feeder roads; (b) rehabilitating 463 basic school buildings; (c) constructing 102 institutional toilets; and (d) conducting tracer studies for rural enterprises. The project closed on June 30, 2011, two years and four months (50%) after the original closing date.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High
The project's objectives are relevant to the World Bank Country Assistance Strategy (CAS) for Ghana. Under the third pillar of the CAS for Fiscal Years 2008-2011, the Government of Ghana and its joint assistance strategy partners committed to support decentralization reforms. The CAS explicitly states that the Community-Based Rural Development Project (the project under review) has been instrumental in piloting local, community-based approaches to service delivery and in advancing the national debate on decentralization (page 15). The CAS states that the lessons from this project should inform the decentralization agenda. The CAS also recognizes the fundamental role of the project in increasing access to safe water in rural areas. It indicates that the project has provided 280,000 people with access to water under the previous CAS and states that, thanks to this project, the objective of reaching additional communities to meet the CAS target of 500,000 with access to water was going to be met.

The project's objectives remain relevant to the Borrower’s strategy. Both the first (2003-2005) and the second (2006-2009) Ghana Growth and Poverty Reduction Strategy (GPRS) papers included among their main objectives increasing expenditure on programs targeting the poorest and most vulnerable in society and decrease inequalities and regional economic disparities. Moreover, both strategies aimed at accelerating decentralization to achieve more efficient service delivery and improve citizens' participation in development. The Local Government Service Act of 2003 provided the legal basis for the decentralization strategy contained in GPRS I and reconfirmed in GPRS II. According to this strategy - articulated in the National Decentralization Action Plan - power had to be transferred to the local governments. In line with the decentralization objective, the project was to be implemented by each local Assembly.

b. Relevance of Design:
Substantial
The project development objective is formulated in very general terms ("strengthen the capacity of rural communities to enhance their quality of life"), although the 'quality of life' was to be improved under three different profiles - increase in productive assets, improving of rural infrastructure, and higher access to services.

The community development approach is based on the assumption that involving communities in project design and implementation enhances community capacity, increases local ownership, and improves project's outcomes. The PAD states that, to achieve poverty reduction all districts, area council and community development plans need to be developed in a participatory manner and that micro-projects should be chosen by the beneficiary communities. The causal chain is not made explicit, though, and the presumed links between greater community participation and projects that are needed, well-conceived, well implemented, and reaching the poor are not thoroughly discussed. For example, there is not much discussion on how to track community participation, formulation of priorities, selection of sub-projects to minimize inefficiencies, elite capture, and corruption, and to achieve correct targeting. However, some of the elements that are known to strengthen Community-Driven Development projects feature in the design, such as government support as a pre-condition and, in particular, the creation of strong local management capacity to maximize accountability and transparency, and ongoing maintenance and management.

The PAD makes a clear case for the value added of the rural enterprise and learning centers component and the natural resource management component. These two initiatives represent innovations derived from the lessons of the Village Infrastructure Project (VIP), the predecessor to this project (PAD, page 8). The inclusion of the rural enterprise and learning centers was meant to develop skills and knowledge to support the creation of rural farm and non-farm enterprises and hence income generation for the rural communities. The addition of a natural resource management component was prompted by the observation that, while management of natural resources is crucial to ensure sustainable use of the community land resource base, rural communities have little incentive for undertaking this function when it is not built into the project. The added component was meant to provide rural infrastructure to rural communities in order to create this incentive and ensure that community-based management of natural resources and environmental conservation are accomplished through technologies and techniques, such as soil and water conservation and fertility enhancement. The PAD observes that, based on what was learned from the implementation of the VIP, both components should strengthen the possibilities of achieving the development objectives.


4. Achievement of Objectives (Efficacy) :

(a) To strengthen the capacity of rural communities to enhance their quality of life by improving their productive assets.
Modest.
Outputs:
  • The capacity of District Assemblies and Area Councils to fulfill the requirements of the Financial Administration and Internal Audit Acts has reportedly been strengthened sufficiently to enable the entities concerned to comply with financial accountability requirements and maintain asset registers.
  • 78 learning centers (delivering entrepreneurial and business development skills to set up businesses and promote private sector development in rural areas) received loans and grants for about US$ 0.8m to upgrade their facilities and by the end of the project 95% of those funded were using improved facilities.
  • Training in vocational skills was provided to 1,606 under- and unemployed persons, which enabled them to establish their own business.
  • 1,210 beneficiaries (learning centers, learning center trainees, and micro-enterprises) received US$ 3.7m in micro-credits to improve their productive assets.
  • 957 of the 1,210 beneficiaries (79%) were provided with micro-credit to set up their own enterprises, which created 1,622 jobs (not including themselves, as clarified by the project team).
Outcomes:
  • The value of productive assets increased by 27% on average, exceeding the target of an increase of at least 5%. However, the 5% target increase was to be achieved for 70% of supported enterprises. The estimate of a 27% increase was derived from a small sample of 40 beneficiaries, representing 3.3% of all beneficiaries; no information on their representativeness was provided.
  • The revenue of beneficiary enterprises increased by 21% on average. As for productive assets, the target was to achieve at least 5% change for 70% of supported enterprises. Again, the estimate of 21% was derived from the same small sample of 40 recipients.
  • Due to the very small sample size, it is not possible to arrive at any conclusion regarding the distribution of the outcomes or to determine whether the benefits were equally distributed or were concentrated on a minority of rural enterprises.
  • No measure was made of household consumption, expenditure, or income, although these were indicators that were intended to be used as proxies for “quality of life”.
  • There is no baseline value of the number of potential beneficiaries (rural enterprises, unemployed people, etc.) to be able to determine whether the number of beneficiaries reached is high or low. The number of beneficiaries appears low in relation to the average per capita amount received (learning centers: US$ 10,256 each; beneficiaries of micro-credit: US$ 3,058 each).
  • The ICR presents measures of achievement by comparing ex-ante and ex-post values of the indicators. However, there is no way of determining what could have been the trend in the absence of the project (no control group was used to measure the impacts). This raises a problem of attribution, which is not discussed in the ICR.

(b) To strengthen the capacity of rural communities to enhance their quality of life by improving rural infrastructure for agricultural development
Substantial
Outputs:
  • All 454 Area Councils and 170 District Assemblies were trained in Participatory Planning, prepared plans with grassroots participation, and received funding to implement priority projects.
  • These District Assemblies and Area Councils fully adopted participatory planning and prepared participatory plans in accordance with National Development Planning Commission guidelines.
  • 469,151 people (283,582, or 60%, men and 185,569, or 40%, women) benefited from various capacity building activities funded by the project, including those related to identification, assessment, construction and management of sub-projects.
  • All District Assemblies and Area Councils were reported to be undertaking procurement in accordance with the National Procurement Law (for example, preparing procurement plans, carrying out evaluations, maintaining proper records, and addressing contractual issues in a timely manner, setting up of tender commissions and review boards).
  • 48 Area Councils received training in the Rapid Results Initiative meant to enhance their capacity to implement projects successfully in 100 days.
  • 195 feeder roads with a total length of 1,277 kilometers were rehabilitated.
  • 1,610.9 hectares of the critically degraded watershed areas have been planted, representing 55% of the critically degraded watershed areas in the 50 Area Councils (in 35 District Areas) where this activity was undertaken.
  • 25 market centers and 14 slaughter houses have been completed.
  • 3 dams, 2 wind pumps, and one dugout were completed. At project closure the 3 dams and 2 wind pumps were not being used for farming, but only to pump water for domestic purposes.
  • More than 500 community members received training in bushfire control and management techniques and were equipped with small fire-fighting tools.
Outcomes:
  • The number of vehicles plying the roads on a market day increased from 3 to 9 on average over 62 completed roads, meeting the target of at least a 200% increase.
  • The average number of days that roads were impassable declined from 173 (average for 40 sampled roads) to 15 per year (calculated from a sample of 62 completed roads).
  • Time spent traveling decreased from 15 to 4 minutes per kilometer on average (a reduction of 73.3% essentially meeting the target of at least 75% decrease).
  • Over 283,880 people (124,591 men, or 44%, and 160,320 women, or 56%) benefited from the improved roads (the project team clarified that this is the total population of the communities served by the improved roads, i.e. the communities that use the improved roads).
  • About 500 people each day had access to water for farming and domestic purposes and 1,490 animals to drinkable water thanks to the dams and wind pumps constructed by the project. These numbers are higher than the targets of 365 and 1,080 respectively. The baseline is not known.
  • The targeted farmers experienced an increase in cropping intensity from zero to twice per year on average, meeting the target (the project team clarified that the baseline of zero refers to farmer beneficiaries entering the business for the first time; essentially new farmers were able to achieve two crops per year on average by the end of the project. No result is reported for those who were already farming.
  • No evidence is presented of changes in food production among target farmers or of increase in revenues obtained by District Assemblies.
  • Other than the amount of training delivered, and plans prepared and implemented, there is no documentation of local community participation in the identification and implementation of project-supported activities.

(c) To strengthen the capacity of rural communities to enhance their quality of life by improving access to key support services from private and public sources
Substantial
Outputs:
  • 458 primary schools were constructed or rehabilitated. More than 60,000 students per school year moved from open air schooling to proper premises.
  • 1,676 water points (boreholes) were constructed.
  • 271 Community-Based Health Planning Services compounds have been constructed and at least 60 are operational.
  • 5 nutrition centers were constructed.
  • 8,910 communities were trained in using two accountability instruments: Community Accountability Fora and Community Score Cards. As a result, 454 ACs held at least two Community Accountability Fora. 167 District teams and 162 Area Councils were trained to use Community Score Cards, which were used to assess the quality of public services in 339 communities.
Outcomes:
  • The number of children enrolled (in primary schools, as clarified by the project team) increased from 50,857 at baseline in the year 2006/07 to 63,216 at the end of the project in year 2010/11 - or by 24.3% total (number calculated for 280 schools).
  • The number of school days cancelled because of bad weather decreased from 15 to zero per year (meeting the target of 99% reduction).
  • The number of people with access to potable water in rural areas increased from 117,711 to 421,315 in 453 communities - an increase of 257.9%, exceeding the target of at least a 200% increase over the baseline. 47% (198,019) of those benefiting are male and 53% (223,297) are female (this is the composition of the population residing in the beneficiary rural areas).
  • The average time elapsing between breakdowns and repairs of water facilities decreased to 1.5 days on average (below the 2 day target).
  • 237,282 beneficiaries accessed health facilities (Community-Based Health Planning Services) at project closure, from 83,840 at baseline (an increase of 183%), meeting the target of 40% increase.
  • No health outcome has been reported (although the efficiency calculations were based on assumptions on the reduction of sick days to be achieved for the beneficiaries - see Section 5 below).
  • According to Community Accountability Fora held in the Area Councils 90% of community leadership was fair and responsive to social accountability issues (the ICR does not report the number of fora for which this percentage was calculated).

5. Efficiency:

Net present value, benefit-cost ratio, and economic rates of return (ERR) were estimated at closure for learning centers and schools. These two categories of investments together accounted for about 30% of total project costs. In the case of learning centers the net present value and the economic rate of return were also calculated at appraisal. In addition, the net present value was calculated for rural health infrastructure (5% of total project investments).
The estimated ERR for schools at closure is 27%. As far as investments in rural enterprises are concerned (learning centers and new businesses established by learning center trainees and small rural enterprises) the economic rates of return are 42% for learning centers (compared with 28% estimated at appraisal), 159% for new businesses started by learning center trainees, and 65% for small rural entrepreneurs.

Some of the assumptions used to calculate efficiency measures seem overly optimistic:
  • The benefits from additional/improved school infrastructure rest on the assumption that the additional students who graduate would be a stable number over time (i.e. the higher attendance will continue), and that they would alll be fully employed for 25 years at 4 cedis per day (US$ 1.00= Ȼ18,932 on December 6, 2012). No account is taken of the different probabilities and wage rates for boys and girls.
  • Potential congestion effects (overcrowded classrooms, for example) or restrictions in supply (of good teachers, doctors, nurses) are not discussed and are not accounted for in the calculation of the ERR;
  • For learning centers and rural enterprises, the ICR does not mention the assumptions made in the calculation of the net present value and the economic rate of return. For learning centers, the ICR appears to rely on the results of a survey among business owners that upgraded their businesses into learning centers and reported increased managerial skills, income, and employment for other community members. The ICR reports that 62 additional working days for hired labor were created on average from converting a business into a learning center, but there is no indication of how sustainable this increase may be over time, or whether the economic value used of a day of family labor (Ȼ 10.21 or US$ 0.54 on December 6, 2012) is plausible and valid for every learning center.

On the other hand, only private benefits (in the school example, the earnings of additional graduates) are considered in calculating the ERR, when in fact the type of investments supported by the project have strong positive externalities (for example, education has a number of benefits besides increasing private earnings).

For rural health infrastructure (5% of total project investments), the ex-post analysis of efficiency identified the reduction in the number of sick days of those visiting the center required for the net present value to move from negative to positive values. It was calculated that the cost of improving health services plus the costs of managing the renovated health centers (maintenance, administration, salaries, equipment, supplies, etc.) equal 3.3 days of work per patient per year over 25 years. This implied that a patient's sick days had to be reduced by 2 thanks to the improved health infrastructures for the net present value to move from negative to positive values. More days saved would make the improved health services economically beneficial to the entire community. There is no discussion in the ICR regarding the plausibility of the assumptions (for example, whether the economic value of an additional working day is applicable to all patients irrespective of their demographic characteristics, and similarly for the length of their working life, assumed to be 25 years). On the other hand, this calculation does not take into account the social and ethical benefits that an improved health center may have. The ICR does not report whether the reduction in sick days was actually achieved.

There were a number of operational and administrative inefficiencies. About 250 entrepreneurs who received micro-credit (more than US$ 3,000 each on average, or US$ 3.7million for a total of 1,210 entrepreneurs) did not start any enterprise; the dams and the wind pumps constructed were not used for farming, but only to pump water for domestic purposes. Twice (2004 and 2008) there were important delays after the national elections in appointing the District Chief Executives and for them to be fully operational afterwards. Moreover, in 2009, the government halted the execution of all contracts in the country for about 3 months. The project closing date was extended for two years and four months, of which 16 months were attributed to the additional activities supported by the Additional Financing.

Efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of objectives is rated high and that of design substantial. Most planned activities were undertaken. Efficacy of two of the three sub-objectives – to strengthen the capacity of rural communities to enhance their quality of life by improving rural infrastructure for agricultural development, and by improving access to key support services from public and private sources – is rated substantial, while that of the other -- to strengthen the capacity of rural communities to enhance their quality of life by improving their productive assets -- is rated modest. Efficiency is rated modest – although economic rates of return were high, they represented less than a third of project costs and are based on overly optimistic assumptions. Overall outcome is assessed as moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The sustainability of the project’s achievements hinges on the capacity of the communities to continue managing and maintaining the infrastructures funded by the project, as well as on Districts and Area Councils retaining their capacity to deliver acceptable quality services.

The ICR reports that the project's investments in building and rehabilitating agricultural and social infrastructure were "well targeted, relatively inexpensive, and required only modest costs for operation and maintenance" (ICR, page 19).

The project supported the creation of facility management committees at the beneficiary level and trained those committees to operate and maintain their facilities. Maintenance would be needed at discrete intervals, as the roads or school facilities deteriorate, so it is not yet possible to assess to what extent full maintenance needs will be provided for. According to the project team, no project has yet been closed because of inability to maintain it.

The project adopted measures to decrease the financial risk by placing qualified financial management experts in all project regions to increase capacity to plan and manage projects at the regional, district, and local levels. This is meant to ensure the availability of funding for operations and maintenance. The ICR does not, however, specify whether or how the qualified financial management experts will continue to be funded after project closure.

In addition to the risks acknowledged by the ICR, there is a political risk stemming from the turnover of District and Area authorities and officials, for example following elections. New appointed officials who do not receive any training in the activities supported by the project may have a poor understanding of the rules and processes established by the project, little appreciation of the importance of M&E, and little commitment to continue project activities (monitoring and ensuring maintenance of infrastructure, participatory processes, etc.). The ICR (page 8) indicates that this risk has already materialized: "although the project built capacity among administrative staff in the DAs and ACs, most could not be retained owing to frequent turnover and staff rotations."

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
The project adopted a Community Driven Development approach and included several activities to strengthen the capacity of local communities to implement projects, such as extensive training.

During the conceptualization stage, the project team produced a number of studies, including a Beneficiary and Stakeholder Needs Assessment for Community-based Natural Resource Management; an Assessment of Rural Water Infrastructure Needs; and a Ghana Social Assessment study.

Design benefited from the experience of the prior Ghana Village Infrastructure Project (implemented from 1998 to 2004), which piloted empowerment of rural communities and good governance by training local councils to prepare participatory plans and manage public goods. The Village Infrastructure Project had received a satisfactory rating by IEG (then OED). The project under review strengthened the positive elements of this earlier project (community involvement in designing, implementing, and monitoring sub-projects), while addressing some aspects that were less effective (for example, insufficiently decentralized financial management, procurement, and M&E functions). It also introduced two new components to further strengthen the project (the rural enterprise and learning centers, the community-based integrated natural resources management, and a wider menu of sub-projects to meet popular demand from the communities, including social infrastructure.

The project team also visited similar projects in Thailand and Indonesia and incorporated lessons from those experiences. For example, the learning centers - one key component of the project - were modeled along the lines of Thailand's Social Investment Fund. The Indonesian experience showed the importance of strengthening sustainability by building capacity of local and already existing institutions rather than creating new ones.

Design also included measures to manage risk, such as the placement of qualified fiduciary experts in the regions where the project operated, which the ICR (page 20) indicates was a "key innovation that aided the project's successful implementation.”

The Bank team included a varied range of skills (agricultural, social development, public sector, environmental specialists) to cover the project's technical and institutional requirements.

There were some weaknesses in M&E design (see Section 10 below).

The PAD reports that an Environmental and Social Management Framework and a Resettlement Policy Framework were prepared in accordance with safeguards policies (see Section 11 below).

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:
Supervision missions were regularly undertaken, including a mid-term review to assess progress and adopt corrective measures. The ICR reports that the Bank’s project team and that of the Agence Française de Développement (that was co-financing the project) included the right mix of skills for ensuring smooth implementation.

The ICR flags a number of implementation issues without clearly identifying the source of the problem or responsibility for it. For example, it indicates that there were instances of incorrect and inadequate filing of documentation that impeded implementation of community projects (page 8) and that, because the project was not designed to facilitate logistics for district staff, the effectiveness of supervision in some areas was undermined (page 8). There is also reference to "a lack of clarity in procedural arrangements between the project, Apex Bank, and Rural Banks, which delayed implementation of the Rural Enterprise Development/Learning Center Component" (page 7).The Natural Resource Management component was delayed because of limited knowledge and skills on the part of the project and district staff, the lack of a focal point at the district level, and unclear institutional arrangements involving partners not previously known to the project (page 8). It is not clear from the ICR how proactive the Bank team was in resolving these issues in a timely manner, although there is a statement (page 20) to the effect that the team was effective in identifying causes of delays in implementing the Natural Resource Management component and recommending actions for improvement at the time of the mid-term review.

A joint World Bank-Agence Française de Development mission in October 2007 assisted the Government in responding quickly to the destructive flooding in Northern Ghana, and facilitating an increase in project financing.

Some, but not all, M&E design weaknesses were addressed during implementation (see Section 10 below).

According to the ICR, the project fully complied with those safeguard policies that were triggered (see Section 11 below).

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
Although the ICR (page 21) reports that the Government’s commitment to the project was high, the counterpart funding, at US$1.41 million, was considerably less than the US$ 8.93 planned at appraisal. The project team explained that this was due to a change in the ability of the government to contribute to the project due to macroeconomic circumstances.

There were delays in implementation due to national elections in 2004 and 2008. More specifically, because of the elections, it took some time to appoint new District Chief Executives, to whom the project assigned an important role in decision making at the level of beneficiary districts. Once the new Chief Executives were appointed, it took further time before they became familiar with procurement laws and contract management guidelines. Implementation was also delayed for about three months in 2009 when the Government issued directives to halt execution of all contracts in the country. This issue was resolved when an exemption was obtained for Bank-financed operations. Finally, the Government’s public financial management systems at the district level were shown to be generally weak (ICR, page 10).

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
The ICR (page 21) reports that the staff of the Project Coordinating Unit (PCU, housed in the Ministry of Local Government and Rural Development) was highly committed to the project and participated fully in project preparation. During implementation, the PCU placed its experts close to project sites, which facilitated planning, financial management, procurement, and participatory M&E. The Unit organized and coordinated supervision missions, and established good working relations with the Government. The Unit maintained proper records and reporting systems for the project’s fiduciary and physical progress, managing and disbursing all project funds satisfactorily. The project was judged to be one of the best performing projects in a World Bank Country Portfolio Performance Review in 2009.

However, the Unit’s lack of expertise in natural resource management (NRM) favored implementation of the other four components in relation to the NRM component, which suffered important delays because of lack of focal points at the district level and unclear institutional arrangements.

There were also cases of incorrect filing of documentation that impeded implementation of community projectspage, as well as lack of coordination between the project, Apex Bank, and Rural Banks, which delayed implementation of the Rural Enterprise Development/Learning Center Component.

The PCU did not engage in an effective dissemination of the project results to the general public page.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The indicators included in the Monitoring and Evaluation Framework mostly reflected project outputs but did not sufficiently reflect the project development objective (in terms of outcomes). Although the original Results Monitoring Framework did include some outcome variables (such as “quality of life as measured by income, access to water, education, and health facilities,” and "the percentage of formerly under-employed and unemployed rural beneficiaries able to establish their own businesses by 2007" ), these were dropped when the M&E framework was revised at the time of the Additional Financing. No explanation is given of why these outcome indicators were dropped.

Indicators of participation of communities in designing, implementing, and managing projects were never included in the M&E design.

The ICR indicates (page 9) that an M&E manual was developed to support data collection.

Several baselines were incorrectly assumed to be zero. The project team clarified that it was difficult to determine the baseline for most of the activities - especially because the project focused on demand-driven activities and therefore it was not possible ex-ante to define the target beneficiaries. Consequently it was agreed that baseline studies would be carried out for each beneficiary group once their proposal was accepted. The baseline was designated as zero based on this understanding. New baselines, however, were not reflected in the ICR, and many remained at zero.

The Project Coordinating Unit was given responsibility for managing the M&E framework.

b. M&E Implementation:
The M&E framework was revised as part of the processing of the additional financing in May 2008, with the aim of introducing more outcome indicators to replace and integrate output indicators. However, the indicators remained largely focused on outputs, while some outcome indicators were dropped, as discussed above. In particular, as noted when discussing efficacy, there are no indicators that adequately measure the ability of local communities to implement their plans or manage and deliver services to local communities.

Data were collected using a sample approach, which facilitated the implementation of the M&E framework. There were, however, problems in M&E implementations due to the fact that the project team followed the approach specified in the M&E manual to collect data, while the District Assemblies used the Government's reporting system. The ICR (page 9) reports that this inconsistency caused delays.

Training and the manual were insufficient to lead to a replacement of established practices by new and better ones. The issue was addressed by collaborating with the National Development Planning Commission at the central level to harmonize district-level M&E by developing district M&E guidelines. The National Development Planning Commission used these guidelines to train district staff to improve the ability of District Assemblies to prepare and implement M&E plans. However, little information is provided on the results of this effort.

The ICR (page 9) notes that the project trained Regional Coordinating Council staff, who trained district teams (who in turn trained beneficiary communities) in participatory M&E. However, there is no information on which participatory methodologies were used or how they complemented other data surveys. The ICR (page 17) reports on the findings of a beneficiary assessment survey, which is plausibly part of the participatory M&E, but this is not explicitly indicated. Moreover, the reporting on the survey results is incomplete, and no indication is provided on the methodology adopted.

A Beneficiary Assessment was conducted, but only very general findings are reported in the ICR, with no detail on methodology, sampling, and specific indicators.

a. M&E Utilization:
There is no mention of which indicators continue to be used to manage local project implementation. The ICR reports that communities trained in participatory M&E techniques continue to use them to implement their sub-projects, which increases the quality of those projects and community ownership, but these claims are not substantiated by evidence.

The ICR (page 22) comments that the Project Implementation Unit did not widely disseminate the findings from the project.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The project was classified as Category B for Environmental Assessment purposes. Two safeguard policies were triggered: Environmental Assessment (OP 4.01) and Involuntary Resettlement (OP 4.12).

The PAD reports that the government prepared and disclosed an Environmental and Social Management Framework and a Resettlement Policy Framework to address any potential environmental, social, or resettlement issues. The Resettlement Policy Framework established the resettlement and compensation principles, organizational arrangements and design criteria to be applied to meet the needs of the people who may be affected by the project.

The ICR is vague in describing how these frameworks were applied; in particular, it does not clearly state whether individual sub-projects regularly prepared individual sub-project resettlement and/or compensation plans.

The ICR (page 10) states that "at mid-term the project had complied fully with safeguard policies, and no serious adverse impact was identified," that land acquisitions occurred within the rules and in compliance with OP 4.12., and that no involuntary resettlement, displacement, or relocation occurred as a result of the project.

The ICR also reports that the project teams developed a good working relationship with the Environmental Protection Agency (EPA) and supported EPA to provide training to all participating District Assembly staff in environmental screening and social issues.

b. Fiduciary Compliance:
Financial Management: The Government’s public financial management systems at the district level were shown to be generally weak (ICR, page 10). The ICR does not discuss how this weakness manifested itself, or what steps were undertaken to address it. The project submitted timely annual audit reports, which had unqualified audit opinions and were acceptable to the Bank.
Procurement was initially an issue. Poor capacity and poor understanding of World Bank procurement rules were evident, especially at the district level. The issue was addressed by placing procurement specialists in all regions and building capacity in public procurement in decentralized units. There were no reported cases of misprocurement.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:
None



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
Relevance of objectives is rated high and that of design substantial. Most planned activities were undertaken. Efficacy of two of the three sub-objectives is rated substantial, while that of the other is rated modest. Efficiency is rated modest – although economic rates of return were high, they represented less than a third of project costs and are based on overly optimistic assumptions. 
Risk to Development Outcome:
Negligible to Low
Moderate
Some of the risks to development outcome were rated "moderate" in the ICR (page 19), although the ICR did not factor in the risk related to maintaining capacity at the District and Area level. An important part of the project consists of capacity building, but this capacity can easily be depleted because of staff turnover, an occurrence that was already manifest by project closure. 
Bank Performance:
Satisfactory
Moderately Satisfactory
There were moderate shortcomings in Quality of Supervision (see Section 8b above). 
Borrower Performance:
Satisfactory
Moderately Satisfactory
While there were moderate shortcomings in Implementing Agency Performance, Government Performance suffered from significant shortcomings, including a considerably lower than planned counterpart contribution and important delays resulting from turnover and lack of appropriate skills in key positions. 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Lessons 1 and 2 are taken (with adaptation) from the ICR, and lesson 3 is drawn by IEG)

For community-based development operations, supporting the decentralization of the project management functions up to the zone and regional levels by using skilled personnel can strengthen smooth implementation of the project. The project’s success hinged on its ability to create managerial and technical capacity at the local level. Placing specialists (accountants, procurement specialists, M&E specialists, engineers, etc.) in all the project's regions allowed for a closer monitoring and supervision of the project activities and strengthened the capacity building component.

Implementation of activities for which there is low capacity, low awareness, and low demand on the ground may require specific, ad-hoc strategies. Component 3 of the project (Rural Enterprise Development and Learning Centers) posed challenges due to inexperience among rural banks and project staff with lending, and to lack of business skills among clients. The project team indicated that a 'pilot approach' - where a few enterprises would have been identified and promoted to start with, before scaling-up - could have been more effective. Similarly, component 5 (Community-Based Natural Resource Management) was delayed because of lack of capacity and ownership at the district and community level. The project team recognized that a special effort should have been made in this case, not only to develop a training manual to increase the capacity of district staff in environmental and natural resources, but also to create general awareness of the importance of natural resource management at the community level.

Quality M&E data are especially important in Community Driven Development (CDD) projects.
While M&E is important in all projects, it is particularly so in CDD operations, which are broad and complex. Moreover, such projects rest on assumptions and conditions that are normally not measured, partly because it is difficult to do so (such as participation of the communities in setting priorities, avoiding elite capture and corruption, good targeting and ability to reach the poor). In addition to devoting a lot of attention to set up a proper M&E system, this type of project would benefit from tracking intermediate indicators capturing, for example, characteristics of the beneficiaries and beneficiary communities; level of local involvement; sub-project screening procedures. Moreover, objectively assessing the achievement of the project's efforts requires that appropriate efforts should be made to identify and measure final outcome indicators. In the case of this project, several important outcome indicators could not be reliably measured (for example, changes in food production; distribution of the benefit from increased turnover of the enterprises) and, in other cases, no data were collected (on, for instance, the quality of the services and infrastructure that had been created, the ability to meet the increased demand through an adequate number of teachers, nurses, and doctors).

14. Assessment Recommended?

Yes
Why?
To verify the ratings and document lessons learned.

15. Comments on Quality of ICR:

The ICR is satisfactory, but barely so. It is reasonably thorough and contains enough information and analysis to enable an evaluation of the project. There are, however, a number of shortcomings. The most significant is the lack of clear evidentiary support for several important statements - for example, "Business ownership significantly boosted beneficiaries' incomes" (page 15); "The provision of CHPS improved key health indicators such as the number of births attended by skilled staff, the maternal mortality rate, the prevalence of contraceptive use, and incidence of diarrhea" (pages 15-16); "improved water quality helped to reduce such waterborne diseases as guinea worm and diarrheal illnesses" (page 16).

There are also some inconsistencies, contradictions, and mistakes in reporting outputs and outcomes. For example, there are discrepancies between the figures reported in the ICR text, those reported in the Annexes, and those reported in the results matrix. A Revised Results Framework and Performance Indicators (as of May 31, 2011), subsequently provided by the project team, contained information at times conflicting with earlier documentation. For example, the ICR text, page 13, states "gross enrolment increased from 20% at baseline to about 41% by the end of the project"; Annex 2 (page 38) states "gross enrolment increased significantly from the baseline of 20% to 29% at the end of the project"; the (more recent) Revised Results Framework and Performance Indicator reports that gross enrolment increased from 50,857 at baseline in the year 2006/07 to 63,216 at the end of the project in year 2010/11 - or 24.3% total (number calculated for 280 schools). The number of markets constructed is 39 according to Annex 2 (page 26) but 25 according to Indicator 19 reported in the matrix of PDO indicators.

In other cases the ICR reports figures without a clear explanation of what they represent. For example, it states that over 283,880 people (124,591 men, or 44%, and 160,320 women, or 56%) benefited from the improved roads (page 27), but offers no explanation of what these figures represent or how they were calculated. The use of percentage changes and percentage points variations is sometimes confused.

The reporting on the Beneficiary Assessment is sparse and includes no details on the methodology used.

The ICR also has contradictory figures for the project costs between the text and Annex 1.

Finally, the ICR states that the Government’s public financial management systems at the district level were generally weak (ICR, page 10), but it does not discuss how this weakness manifested itself, or what steps were undertaken to address it.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions