Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Bn-national Community Driven Development Project


  
1. Project Data:   
ICR Review Date Posted:
09/18/2013   
Country:
Benin
PROJ ID:
P081484
Appraisal
Actual
Project Name:
Bn-national Community Driven Development Project
Project Costs(US $M)
 65.87  79.90
L/C Number:
C3990, CH128
Loan/Credit (US $M)
 50.55  63.85
Sector Board:
Social Protection
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  10/07/2004
 
 
Closing Date
05/30/2010 04/30/2012
Sector(s):
Other social services (26%), Health (26%), Primary education (26%), Sub-national government administration (13%), Micro- and SME finance (9%)
Theme(s):
Decentralization (33% - P) Other human development (17% - S) Education for all (17% - S) Participation and civic engagement (17% - S) Other social protection and risk management (16% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Elena Bardasi
Judyth L. Twigg Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:


    According to the Project Appraisal Document (p. 4) and the Financing Agreement (p. 19), "the objective of the Project is to promote the use of the community-driven development approach by line ministries, decentralized local governments and local communities to improve the access of the poorest communities to basic social and financial services, so that the line ministries and decentralized local governments will be ready for further implementation of community-driven development (CDD) activities through programmatic support."

    This original objective can be broken down into two outcome-oriented objectives:
    1. "the line ministries and decentralized local governments [are] ready for further implementation of community-driven development activities through programmatic support."; and
    2. "to improve the access of the poorest communities to basic social and financial services."

    The evaluation of the first objective requires interpretation of what "being ready" means. The rationale provided in the PAD (pp. 1-3), the project design, and the revised objectives at restructuring suggest that this objective can be interpreted as "to increase the capacity of the government to implement the CDD approach."

    The project was formally restructured in June 2010 with the following revised objectives: "to (i) increase the utilization of the community-driven development approach, and (ii) improve access for the poorest communities to basic social and financial services, in the Recipient’s territory" (Financing Agreement [Additional Financing for National Community-Driven Development Support Project], p. 5).

    The revised objectives place emphasis on "increasing the utilization" instead of "promoting the use."

    The original outcome indicators and targets were:
    1. by the end of each fiscal year, at least 90% of IDA funds expected to be managed by targeted communities have been effectively managed by them;
    2. by the end of each fiscal year, 100% of the Borrower's Financing shall have been disbursed to finance sub-projects; and
    3. by the end of the project, the execution time of community sub-projects (from submission to completion) financed with Government Financing is not superior to the execution time of community sub-projects financed with IDA funds.

    The revised outcome indicators and targets (after restructuring) were:
    1. Public capital expenditures on basic services that are implemented by communities through the CDD approach (% of total public capital expenditures on basic services, three year rolling average)(rephrased with respect to original);
    2. The original was dropped because measurement of disbursements is not a suitable outcome indicator;
    3. Moved to intermediate indicator and re-phrased: "Time taken for the execution of community sub-projects financed with Government Financing relative to the time taken for IDA-financed community sub-projects (ratio.)"
    Five new outcome indicators were added:
    4. Direct project beneficiaries (number); of which female (%);
    5. Students enrolled in schools constructed/rehabilitated through the CDD approach (number);
    6. People with access to an improved water source in rural areas (number);
    7. People with access to micro-finance services (number);
    8. Beneficiaries satisfied with the results of the sub-projects (%).

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 06/17/2010

c. Components:

The project had originally four components (that were not revised at restructuring). A fifth component was added at the time of restructuring.

1. Strengthening the capacity of the ministries, communes, and communities to implement community-driven development or CDD (appraisal estimate US$9.44 million, of which IDA credit US$8.18 million; actual US$9.64 million).
This included four main sub-components:
(a) Strengthening the institutional capacity of the public sector at the central and local levels to develop and implement CDD strategy, policy, and programs, by assisting the Government in developing the legal and administrative framework for the implementation of a national CDD policy, and building the line ministries' capacity to realign their roles according to both decentralization and CDD;
(b) Strengthening the capacity of Communes to implement CDD, at the village, neighborhood, and Commune levels, by building the capacity of the participating local governments to integrate a CDD approach into their local planning and the implementation mechanisms of sub-projects according to the subsidiary principle;
(c) Strengthening the capacity of grassroots communities to plan and implement village and neighborhood sub-projects within their Commune Development Plans (PDC -- Plan de Developpement Communal), by building the communities’ capacity to initiate, develop, implement and monitor sub-projects;
(d) Strengthening the capacity at all levels to provide M&E and communication on project-related issues to all stakeholders. This sub-component was meant to support the installation and operation of an M&E system to facilitate management and strategic decisions at all levels and the establishment of a communication system to ensure transparency and improve accountability.

2. Improving access by the poor to basic services and infrastructure (appraisal estimate US$44.29 million, of which IDA credit US$18.86 million and IDA grant US$12.30 million; actual US$57.02 million).
This component was meant to:
(a) Improve access of the poor to basic services and infrastructures through multi-village sub-projects implemented by Communes, by providing grants to the Communes for the implementation of communal sub-projects of multi-village interest. The sub-projects to be funded had to be initiated by the Communes themselves, included in the Commune development plans (PDC), developed according to sectoral norms and specifications, and implemented by the Communes themselves.
(b) Improve access of the poor to basic services and infrastructures through single villages or neighborhood sub-projects implemented by communities themselves, by providing grants to communities for the implementation of small-scale sub-projects of single village interest, initiated and prepared by the targeted communities in a participatory manner on the basis of their priorities. These sub-projects had to be approved and financed by the Communes in the framework of their PDC, but their implementation was the complete responsibility of the communities themselves, including financial and procurement management.

3. Improving access of the poor to financial services for income-generating activities (appraisal estimate US$5.43 million, of which IDA US$5.15 million; actual US$2.95 million).
This component was meant to:
(a) strengthen micro-finance institutions' (MFIs’) capacity to supply microfinance products and services better matching the needs of targeted communities, with special attention to financing income-generating activities. Project activities under this sub-component would provide grants to existing MFIs to fund acceptable plans for the development of new financial products and services that better suit the needs of the project’s target population seeking access to financial services, including loans to finance income-generating activities;
(b) provide advisory services for the development and management of income-generating activities. This sub-component was to provide grants to communities to finance the advisory services of their choice to improve their income-generating activities, mainly in agriculture, with the possibility to be financed by MFIs.

4. Support for the management of community-driven development (appraisal estimate US$6.05 million, of which IDA US$4.85 million; actual US$9.75 million)
This component was meant to:
(a) support the Executive Secretariat (Secrétariat Exécutif Permanent – SEP) of the National Committee for Community-Driven Development through the provision of technical advisory services to enable it to coordinate the day-to-day operations for project implementation, manage IDA and Government counterpart funds, support the management of Government financing, and prepare the transition to programmatic support;
(b) support Environmental Management. This sub-component was to provide (i) a mechanism to screen future sub-projects for potential environmental and social impacts and identify needed protective measures; (ii) environmental training for members of the SEP as well as the Fiduciary Agency (FA), Communes, concerned line ministries and NGOs, and representatives of the beneficiary communities; and (iii) an outline of the institutional arrangements under the project that would allow for the effective implementation of sub-projects as well as the strengthening of environmental management capacity in the context of Benin’s ongoing decentralization process.

The fifth component, added at the time of restructuring, was:

5. Technical assistance for development of an efficient Social Safety Net (SSN) system (appraisal estimate US$0.50 million; actual US$0.25 million)
This component was meant to:
(a) carry out a technical assistance program to strengthen the Recipient's SSNs, including (i) a review of the existing SSNs; (ii) a review of the Recipient's needs in terms of SSNs, updated information on possible target groups for SSNs, and definition of appropriate instruments for each such target group; and (iii) the preparation of an action plan to improve the cost-effectiveness of existing SSNs and assist with the development of new SSNs; and
(b) carry out a technical assistance program to develop a Cash Transfer Program in the Recipient's territory, including (i) feasibility studies specifying design parameters for a Cash Transfer Program (i.e. potential coverage, benefit levels, methods of targeting, monitoring and evaluation mechanisms, etc.); (ii) studies/assessments on an appropriate implementation arrangement for such Cash Transfer Program, including institutions involved in the implementation of the potential pilot program; and (iii) the development of a staff capacity building plan for the proper management and administration of the potential Cash Transfer Program.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project costs
The project costs at closing were aligned with planned costs (after restructuring), although funding was redistributed across components (from the safety nets and the financial and income-generating activities components to the fourth component, support for the management of community-driven development).

Financing
The project cost at appraisal included US$12.20 million in IDA grant financing (under the second component; actual US$13.07 million) and US$3.02 million to be provided by contributions of the local communities (actual US$2.98 million). US$12.00 million additional financing was approved in June 2010 under the Crisis Response Window.

According to the disbursement profile in the ICR Data Sheet, both the original Credit and the Additional Financing were fully disbursed.

Borrower
The project benefited from a contribution by the Government of Benin of US$12.5 million, plus an additional US$2.0 million to fund more local investments using Government financial channels.

Dates
The project was approved by the World Bank Board of Directors on October 7, 2004 and declared effective on May 2, 2005.
At restructuring, the project's original closing date (May 30, 2010) was extended to September 30, 2011.
The project's closing date was further extended to April 30, 2012 on July 27, 2011, because of delays in implementation.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

High
The project development objectives were and remain relevant to the economic and social conditions of Benin. In 2003, at time of project appraisal, the national poverty rate was 39 percent, with 22 percent living in extreme poverty. Poverty was disproportionately concentrated in rural areas and manifested in lack of access to basic services, particularly education, health, and water. The Government's strategy to improve services to the poor was progressively building on the decentralization process that had started in 1999, and relied on increasing local control and community participation in the management of services.

The project's objectives are fully aligned with the Government Strategic Development Orientations (SDO), which defined the development policy framework for 2006-2011 and whose second fundamental objective is "reducing poverty and improving the quality of life of the population" (as reported by the Country Assistance Strategy, 2009, p. 6). The Growth Strategy for Poverty Reduction - adopted in April 2007 as Benin's second-generation Poverty Reduction Strategy Paper - is based on the main elements of the SDO. The project's objectives are fully aligned to its second pillar ("infrastructure development"), third pillar ("developing human capital and improving basic services"), and fourth pillar ("improving governance and transparency and fighting corruption").

The project's objectives are relevant to the World Bank's strategic support to Benin, both at time of the project's closure and at time of the project's design. The Country Assistance Strategy (CAS) objectives (at design) included the elaboration of a national CDD support project to build on the experience of three successful World Bank-funded community-level operations (PAD, p. 3). The CAS for Benin for the period FY09-12 (the latest approved strategy document) has as a second strategic objective "improving access to basic services," including improving access to safe and sustainable drinking water and sanitation services, improving quality education services, and increasing access to health services -- all main goals of the project. Moreover, the third strategic objective of the CAS is "promoting better governance and strengthening institutional capacities," including support to decentralization through community development. The project is also fully relevant to this strategic objective, which states that "better implementation of the decentralization program is needed to strengthen accountability, transparency in local decision making, and responsiveness to local concerns" (CAS, 2009, p. 20).

b. Relevance of Design:

Substantial
The project design emphasized the need of both responding to one of the main challenges facing Benin, that is improving access to services for the poor, as well as supporting the new decentralization framework adopted by the Government. The PDOs are consistent and clearly articulated around those two priorities.

The rationale of the design was strongly anchored on the Community-Driven Development approach: support communities in the identification and implementation of projects using participatory mechanisms in order to reflect priority needs, while at the same time ensuring lasting institutional effects on line ministries, local governments, and communities in terms of increasing their capacity to support CDD by the end of the project. The components of the project were coherent with these two goals and especially emphasized capacity building at all levels.

The need of having in place the "right" legal and regulatory framework as a pre-condition for project effectiveness was clearly identified. The design strongly emphasizes this element and includes in its first component a number of activities to ensure that the relevant laws, regulations, and institutional bodies are in place and effective.

The support to access to services for the poor was mostly targeted to provision of infrastructure, while the budget allocation for access to financial services and income-generating activities was much smaller. While targeting support for basic services and infrastructure for education, health, water and sanitation is to the benefit of the larger community (and more likely inclusive of the poor), more so than supporting micro-finance institutions and income-generating activities, one may question whether this latter component (especially the income-generating activities part) received the critical minimum level of support to ensure full success, especially in the light of the results achieved (below expectations).

Despite the PDOs inclusion of increased access to social and financial services by poor communities, the original outcome indicators did not measure real impacts on the ground. At restructuring a number of more appropriate outcome measures were selected. Although restructuring happened almost six years into the life of the project and less than two years before its closing, the team was able to retroactively track new and more pertinent outcome indicators -- an indication that the causal chain from inputs to outcomes, even if not fully explicit, was clear and correctly constructed in designing the project.


4. Achievement of Objectives (Efficacy) :

Objective #1 (original): "The line ministries and decentralized local governments [are] ready for further implementation of community-driven development activities through programmatic support.": High

Outputs:
The capacity of the government to implement the CDD approach was built through a series of activities at the national, sectoral, commune, and community levels, and by improving the ability at all levels to provide M&E and communication, as follows:

(at the national and sectoral level)

    • The Law 2001-07 on La Maîtrise d’Ouvrage Public was revised to allow the delegation from communes to communities as required by the CDD approach. The ICR erroneously suggests it was "approved" (p. 26), but it was already in place when the project was approved. The project team clarified that much emphasis was placed in the preparatory phase on the importance of the law to be in place as a condition for project effectiveness.
    • A National CDD Coordination Commission was created as a high-level policy defining and coordinating body.
    • The principles of the CDD approach -- such as participatory planning, subsidiary principle, and the role of the communities -- were explicitly included in the National Guidelines for Commune Development Plans.
    • A National decree in 2009 was issued allowing for the transfer of funds from local government to community accounts.
    • The National Procurement Code adopted in 2009 was more conducive to community-managed procurement practices.
    • A National Operational Directive on CDD in Sectoral Policies was adopted in 2011 to provide consistent guidance for all sectoral ministries on how to integrate CDD approaches.
    • Five key strategies adopted by the Ministries of Health, Primary Education, Finance, Water, and Agriculture incorporated CDD into their operational processes.
    • Tool-kits on service delivery through CDD were developed and disseminated by the same five key Ministries.
    • 240 central staff and 1080 deconcentrated staff participated in 3 training sessions.
    • Deconcentrated staff participated in all stages of the sub-project cycle, including ex-ante reviews and supervision visits.

(at the commune level)
    • Over 8,800 staff in 74 communes were trained in general CDD principles, participatory budgeting, integration of community-driven projects into communal development plans, participatory monitoring and evaluation, and management of the implementation of small-scale infrastructure. 97% of commune staff was satisfied or very satisfied with the training received, based on the results of a Beneficiary Assessment carried out in 2011.
    • Commune capacity increased from a distribution of 31-64-3 percent (low-middle-high) to 0-13-87 percent, as measured by a composite score based on 15 performance indicators, surpassing the target of 0-30-70 percent. The ICR notes that, despite the fact that other decentralization programs were providing training in the same communes targeted by the project, the growth in performance was highest for the indicators directly supported by the project.

(at the community level)
    • A pool of 36 senior and junior trainers was established.
    • 3,000 community focal points were identified.
    • Grassroots Management Training (GMT) was carried out in all communities, with 2,491 training sessions involving 144,495 participants (person-presence) and over 29,000 people trained. Spill-over effects -- each trained participant carried out informational sessions within their communities -- brought the number of those receiving GMT orientation up to 750,000.
    • Test scores of basic competencies on GMT content averaged 84%, based on an ex-post evaluation of GMT. 98% of women and 95% of men indicated that they were satisfied with GMT training.

(strengthening capacity to provide M&E and communication)
    • A communication campaign was carried out, consisting of interactive programs on the project and its community management aspects, through 30 community radio stations in the country. The project supported 1,428 broadcasts in 27 different local languages. Based on the Beneficiary Assessment carried out in 2011, 98% of people in the areas targeted by the project had heard of it; the evaluation of the impact of the radio emission program found that 91% of villagers listened to the emissions. 4 information programs were developed for television, each presented twice on the national television channel. A quarterly project newsletter was distributed to all village development associations, sectoral ministries, international partners, and national assembly (5,000 copies overall). The project website (www.pndcc.org) was developed and maintained, although it has not been updated since October 2011 (IEG visit of 04/26/2013).
    • A number of evaluation studies were conducted, including (i) a Technical Audit of Infrastructure; (ii) an Ex-post Evaluation of the Grassroots Management Training; (iii) an Assessment of Social Safety Nets in Benin; (iv) a Feasibility Study of Cash Transfer Programs for Benin; (v) an Impact Evaluation of the project; (vi) an Evaluation of the Cost Structure of MFIs; (vii) two Beneficiary Assessment Surveys; (viii) an Environmental Audit; and (ix) an Impact Evaluation of the project's local radio program.
    • Training in M&E at the local level was one of the eight Grassroots Management Training modules covered in every beneficiary community (see above, on GMT).

Outcome:
    • At the end of the project, all (100%) Credit Funds were managed by local community beneficiaries, surpassing the target of 90%.

Objective #1 (revised): "Increase the utilization of the community-driven development approach": Substantial

Outputs:
The output indicators for the original objective #1 are also output indicators for the same revised objectives. All the capacity building activities that increased the government's capacity to implement the CDD approach are also those that increased its utilization. One specific indicator was identified as an additional intermediate indicator at restructuring (it was originally an outcome indicator): the time taken for the execution of community sub-projects financed with Government financing relative to the time taken for IDA-financed community sub-projects (ratio). This indicator, measured by the ratio between the average duration (in days) of implementation of sub-projects financed with the Borrower's financing (in the numerator) and financed under a local community grant (in the denominator), was 1.1 at the end of the project (that is, 10% longer for sub-projects financed with Government financing). (The duration was defined as the average number of days between the date of submission of a sub-project proposal and its completion date.) The team clarified that, while not attaining the target (it still took 10% longer for a sub-project to be completed with government resources as compared to IDA resources), this ratio still improved over time, since the first measurement in 2008 put the value at 1.21, or 21% longer for a government-financed sub-project.

Outcome:
    • At the end of the project, 38.3% of all public capital expenditures on basic services were implemented by the beneficiary communities through a CDD approach. This figure (calculated as a three-year rolling average) was higher than the target of 20%.

Objective #2 (original): "to improve the access of the poorest communities to basic social and financial services": Substantial
Objective #2 was not revised, although new outcome indicators were added and several intermediate outcome indicators were re-phrased.

Outputs:
    • The project implemented 1535 community-managed and 113 commune-managed investments targeted to the poorest 1,518 communities (or 40% of all communities) in the country. The total number of individuals who benefited by the project were about 750,000, of which 42% were women. This is above the target of 450,000 and 40% women. Technical audits found that 100% of the infrastructure supported by the project aligned with sectoral quality standards and 90% was classified as functional.
    • The project constructed or rehabilitated 3,170 classrooms, 27% more than the target of 2,500.
    • The project constructed or rehabilitated 101 improved community water points, twice as many as the target of 50.
    • The project built 144 health posts, 20% more than the target of 120.
    • 6 micro-finance institutions (MFIs) -- 1 more than the target of 5 -- received financing to help them expand their services to target communities. Financing paid for training, equipment, and fixed costs of opening up new windows, but did not fund the credit itself. These MFIs were able to extend their services to 512 new communities, surpassing the target of 500, through 14 new branches and 40 new service windows.

Outcomes:
    • The number of students enrolled in schools constructed or rehabilitated by the project was 158,500, 59% more than the target of 100,000.
    • The number of people with access to improved water sources in rural areas was about 25,000, or 2.5 times the target of 10,000.
    • The number of people with access to micro-finance services was about 38,000 at the end of the project, largely surpassing the target of 10,000. The project team clarified that the figure of 14,000 beneficiaries reached reported by Table 2 of the ICR is wrong, in that it only refers to one of the MFIs (CCIF) as of 30 October 2010, and not to the total.
    • 1006 sub-projects supporting income-generating activities (IGAs) have been completed, 29% less than the target of 1,400. The project team clarified that each one of the 1006 IGA sub-projects benefited a community group, although the exact number of people within the 1,006 groups that benefitted from the IGA sub-component is not available.
    • According to a beneficiary assessment carried out in 2011, 94% of the beneficiaries were satisfied with the project. According to the ex-post evaluation of grassroots management training (which included measures of overall beneficiary satisfaction with the project), 96% of the beneficiaries were globally satisfied with the impact of the project.

5. Efficiency:

Efficiency is rated Substantial.
Measures of efficiency are available for parts of the project:

Infrastructure:
An independent technical audit found that the project delivered school infrastructure (primary schools) faster and at lower costs than conventional methods, while ensuring the same level of quality. Expenses for school infrastructure represented about 71% of community investments.
The schools built by contract management agencies with financing from the Bank's Fast Track Initiative Education project were between 40% and 57% more expensive than those built through the project, while being of similar quality. A similar difference in costs was found for the provision of school furniture (desks and benches). The project also built school infrastructure under the Fast Track Initiative at a slightly higher cost than under the CDD approach.
School infrastructure was also delivered faster through the project than under the Fast Track Initiative. The performance of three contractors (two under the Fast Track Initiative, one under the project) in delivering schools during the same time period was compared and, at the end of the observation period, 88% of the classrooms built under the project were more than 75% advanced, while the majority of the classrooms delivered under the Fast Track Initiative were only halfway completed at most. (The project team later clarified that the Fast Track Initiative (FTI) was a separate project that came several years after the start of PNDCC. The Government was impressed with the PNDCC experience and decided to use the PNDCC mechanism to implement one-third of the school construction supported under FTI. This is why the project was also involved in building schools under the FTI.)

MFI
The ICR presents some evidence of efficiency of the microfinance institutions, but only in relation to the private benefit (of the institutions) and not the more comprehensive social benefit. According to an economic analysis of the changes in MFI portfolios, both the savings base and the lending activities expanded. The reimbursement rates of credits were 93%, which indicates that the loans were financially sustainable. There are no data, however, on the benefits derived from these loans for the clients of the MFIs. There are also no data on efficiency for the part of the project supporting income generating activities.

Community training
The ICR documents a progressive decrease of the costs of delivering Grassroots Management Training. The initial unit cost of US$1,500 (for a pair of trainers - one junior, one senior) dropped to US$ 1,280 (for one professional trainer plus one trained community agent) and then again to US$ 855 (two trained community agents supervised by a professional trainer), a 43% increase in cost efficiency over the life of the project. These gains, due to the adoption of a "train the trainers" approach, do not measure efficiency in absolute terms, but they are indicative of gains in efficiency over time.

To sum up, only partial measures of efficiency are available. The measure that refers to school infrastructure, though, was cleverly calculated using a survey that allowed a comparison of two different approaches and different providers, for two different types of schools (with three classrooms and two classrooms, with and without office and storeroom). Expenses for school infrastructure represented about 71% of community investments (the ICR does not provide elements to calculate which exact percentage of the project this item represents, and the project team was not able to provide addition information in this respect).

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Original objectives: Based on High relevance of objectives and Substantial relevance of design, Substantial efficiency, High achievement of the readiness for implementation of the CDD approach objective, and Substantial achievement of the access to social and financial services objective, the outcome based on the original project objectives is rated Satisfactory (corresponding to a rating of 5 on the 6-point scale).

Revised objectives: Based on High relevance of objectives and Substantial relevance of design, Substantial efficiency, Substantial achievement of the increase the utilization of the CDD approach objective, and Substantial achievement of the access to social and financial services objective, the outcome based on the revised project objectives is rated Satisfactory (corresponding to a rating of 5 on the 6-point scale).

The overall project outcome is based on the average of these ratings, weighted by the amount of the total loan disbursed before (82%) and after (18%) restructuring, as follows:

Outcome, original objectives = Satisfactory (5) x .82 = 4.1
Outcome, revised objectives = Satisfactory (5) x .18 = 0.9

Total: 5 or Satisfactory

The project achieved and in many cases surpassed almost all of its targets. There was a minor shortcoming in achieving one target related to the first objective (a decrease in the time it took for a sub-project to be completed with government resources as compared to IDA resources), but there was a consistent improvement during the duration of the project. The second objective was not achieved entirely; in particular, the income-generating activities sub-component did not achieve its target, although the quality of the sub-projects that were completed was very promising. On the other hand, the targets relative to improving access to basic services were surpassed.
While measures of overall efficiency are not available (as is common in community-driven development projects), an extremely interesting study was carried out on the efficiency of school infrastructure provision, indicating very positive results.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The overall sustainability of the project depends on the capacity of the communities to maintain and operate the infrastructure built by the project and their ability to continue to derive benefits from the sub-projects that were created.

The project sought to ensure community engagement in maintaining the infrastructure through the creation of community maintenance committees, and provision of training in basic infrastructure maintenance to communities. Certain types of infrastructure (such as market infrastructure constructed by communes) charge a market fee meant to fund future maintenance. These institutions are still in place.

As for staffing, the ICR indicates (p. 10) that the Government will provide teachers to the additional classrooms under the existing maintenance arrangements, as it is doing under the Fast Track Initiative. The ICR points, however, to the existence of sporadic delays in staffing health projects.

The sustainability of the institutional reforms depends on the Government's commitment to decentralization. This commitment appears to be strong, as indicated by changes in the national legal framework to integrate the community-driven development approach. Guidelines for local governments include participatory planning mechanisms, and ministries have produced guidelines for implementing small-scale investments in their sectors using the CDD approach. The ICR indicates that the capacity created through training is likely to be maintained - given the high level of understanding of training and the ongoing learning-by-doing - but does not discuss if turnover of trained individuals could threaten sustainability.

The achievements of the projects are likely to be strengthened by the follow-up project "Decentralized Community-Driven Services Project (PSDCC)" (US$ 46 million 2012-2016), which will support the construction of small-scale basic infrastructure, pilot safety net interventions, and provide capacity building including grassroots management training.

What is more doubtful is the sustainability of the benefits generated by the support to MFIs and income generating activities. The ICR reports that, at project closure, the MFIs have a very good average reimbursement rate (93%), but it is unclear how sustainable this can be into the future due to the lack of information on how microfinance is used by the beneficiaries of loans. Even less information exists on the income-generating activities (including on current benefits). Neither microfinance nor income-generating activities are receiving support from the follow-up project PSDCC, which suggests that the risk to development outcome of these activities may be higher than for the rest of the project. That said, these activities represent a very small portion of the overall budget.

a. Risk to Development Outcome Rating: Negligible to Low

8. Assessment of Bank Performance:

a. Quality at entry:

The project design drew from previous country experience with community-driven approaches, including three Bank-financed projects (the Borgou Pilot Rural Support Project, the Local Food Security Intervention Project, and the Social Fund Project) that had closed just before the project under review was approved. The lessons from those projects, as well as those drawn from an inventory of 28 other experiences with community-based approaches in Benin, were reflected in the design of the project, especially in relation to modalities of community organization, funds flow management, and inclusion of income-generating activities (as clarified by the project team). The project team also stressed that some aspects, such as the Grassroots Management Training (GMT), were inspired by international experiences, in particular the Senegal CDD. The team also indicated that the major departures from previous CDD projects in Benin were (i) the emphasis on creating a pathway to align the CDD approach with decentralization, and (ii) the move from the tacheronat (local craftspeople coordinated by the ADV) approach to an alternative approach relying on local contractors.

The Bank team took the following steps to minimize a number of risks:
- the creation of a multi-sectoral coordination committee was meant to ensure buy-in and continuity of policy guidance and supervision, and proved effective when the project moved from the Ministry of Development, Coordination and Government Action to the Ministry of Micro-Finance, Youth, and Women's Employment;
- the requirement that the Government financing for sub-projects was disbursed first each year was meant to ensure that Government co-financing was committed and disbursed in a timely fashion;
- effectiveness of financial management was strengthened by contracting fiduciary functions to an autonomous agency (AGeFIB) with experience in managing Bank projects;
- training and capacity building to local governments was meant to prepare competent staff to manage and supervise the sub-projects. At the same time local governments were also given the possibility to hire contract management agencies if this option could offer better results.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:

The team undertook regular supervision missions and developed a score-card of project performance that was reviewed at each mission.

Local fiduciary staff carried out regular ex-post reviews and assessment of performance, including through visiting the field offices of AGeFIB, the agency contracted to perform fiduciary functions.

The project team supported the executive secretariat of the project in managing underperforming local consultants and identifying international consultants to strengthen local capacity.

The Bank team recognized shortcomings in the definition of project indicators and addressed those at restructuring.

There were minor shortcomings in supervising the financial services and income-generating activities component, with an almost exclusive focus on disbursement and outputs, and insufficient focus on outcomes. The team did, however, promptly realize the slow pace of implementation of the income-generating activities sub-component and in successive supervision missions called for the Ministry of Agriculture and the SEP (Secrétariat Exécutif du Projet National d'Appui au Développement Conduit par les Communautés) -- jointly implementing this component -- to accelerate the pace of implementation.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government showed strong commitment to the project by (i) changing the national legal framework to integrate the community-based approach and (ii) providing counterpart funds as planned, plus an additional US$2.0 million to fund more local investments using Government resources.

The project was originally under the Ministry of Development, Coordination and Government Action and was then placed under the Ministry of Micro-Finance, Youth, and Women Employment following a ministerial reshuffling. Despite this rearrangement, the commitment of the Government remained strong.

There were minor delays in the national procurement and audit functions carried out by Government agencies. For example, delays in procurement towards the end of the project originated from delays of up to six months in obtaining approvals and signatures from the National Public Contracting Office.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:

The executive secretariat of the project was SEP (Secrétariat Exécutif du Projet National d'Appui au Développement Conduit par les Communautés), while the fiduciary functions were contracted out to AGeFIB.
Implementation was smooth until June 2010, when the project experienced some delays; however, these delays were made up and the project regularly closed in 2012 having completed all activities (with the exception of the income-generating activities sub-component - see below). SEP was also effective in providing supervision.

The income-generating activities sub-component suffered from delays in implementation, due to delays in preparation of training materials and delays in receipt of applications from communities, approval of proposals, and contracting of experts to provide technical advice to communities. The project team indicated that at times the decision was made to prioritize other components over the income-generating activities sub-component. There were uncertainties about availability of enough funding to complete all planned activities, which caused an implementation freeze toward the end of the project. SEP was convinced that, even with resources from the Additional Financing, not enough resources would have been available to complete all activities, but the project closed with US$540,000 remaining undisbursed -- but at this stage it was too late to re-start implementation of this sub-component.

AGeFIB had previous experience in implementing Bank projects, so there was not a learning curve at the beginning of the project. The ICR mentions that delays and a drop in the quality of performance were experienced towards the end of the project because of key staff leaving AGeFIB (ICR, p. 22).

Overall, the ICR concludes that the performance of the implementing agency was strong as shown by the achievements of the project development objective and by the fact that the main indicators and most of the intermediate indicators exceeded their targets (p. 22).

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

M&E consisted of three components: (i) a management information system that allowed tracking of activities and expenditures; (ii) participatory monitoring and evaluation done at the community, commune and prefectural levels to track progress of the project; and (iii) a set of evaluations of quality and results, including two technical audits, two beneficiary assessments, an ex-post evaluation of the grassroots management training, and evaluations of the activities involving community radio and microfinance. The team built the M&E system on the management information system that was already in place through the AGeFIB agency that had managed the previous Benin Social Fund Project.

Overall, the tools proposed for M&E were in principle quite adequate to monitor project performance and measure achievements of results. In particular, the number of studies planned to measure results was impressive.

b. M&E Implementation:

Training in M&E was extensive and carried out at all levels. Training in M&E was one of the 8 grassroots management training modules covered in every beneficiary community. Because of weak local capacity, the project invested many resources in training in M&E.

Despite those investments and the utilization of an existing core management information system, supervision missions often identified inconsistencies and weaknesses in reporting, due to weak technical capacity for evaluation.

Following restructuring in 2010 it was necessary to include additional indicators in the M&E system that could more appropriately measure the project's impact on the ground. The project team indicated that the new outcome indicators were selected based on the possibility to retrofit with the correct information. While a correct identification of the project's desired outcomes (and corresponding set-up of the M&E tools and indicators) would ideally have occurred at the beginning of the project, the team was able to successfully incorporate meaningful indicators 6 years into the project.

a. M&E Utilization:

Evaluation was strong with respect to the access to basic services component (which accounted for the majority of financing) and the Grassroots Management Training. It was weaker for the micro-finance and income generating activities component.

In the end, all key performance indicators were reported with regularity and all the revised indicators were available at the end of the project.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:

Due to the nature of eligible sub-projects, the project triggered safeguard policies OP 4.01 Environmental Assessment, OP 4.09 Pest Management, OP 4.12 Involuntary Resettlement, and OP 4.37 Safety of Dams, with a safeguard screening category of S2 and an environmental screening category of B.

Training sessions were carried out in all communes on environmental screening for sub-projects, and mechanisms and checklists were developed to implement those screenings.

In terms of process, according to the mid-term review every sub-project completed the environmental screening checklist, but the quality was mixed. According to an environmental and social audit carried out in early 2011, over 95 percent of the communities were aware of the project's environmental guidelines, and 85 percent of community-level respondents indicated that environmental criteria were taken into account in the implementation of sub-projects. Environmental screening sheets were attached to project documentation in 83 percent of the files reviewed during audits.

In terms of actual impacts, field-based evaluations indicated that occasional deficiencies occurred, such as lack of latrines in urban markets. A beneficiary assessment carried out in 2008 found that 92 percent of beneficiaries felt there were no harmful impacts of the projects implemented in their communities; 5 percent noted some (negative) environmental impact. The final technical audit found positive environmental impacts, like improved hygiene conditions from latrines and water projects and improved security from better road access. There were some isolated cases of erosion around buildings and lack of latrines in some schools.

The ICR does not report any finding regarding potentially displaced people, but notes the Beneficiary Impact Assessment carried out in 2011 found that vulnerable people felt useful and respected in their roles in sub-project implementation (p. 42), which indirectly suggests that a sensitive approach was adopted that was attentive to the needs of the vulnerable population.

The ICR does not state directly whether there was compliance with safeguard policies.

b. Fiduciary Compliance:

The fiduciary functions were contracted out to AGeFIB (Agence de Gestion et de Financement des Initiatives de Base). This fiduciary agent was chosen based on its experience and proven capacity to manage Bank projects. Moreover, AGeFIB had properly managed two Bank grants during project preparation. This choice ensured that financial management systems were well in place when the project started. Local fiduciary staff systematically carried out ex-post reviews and assessments of performance, often including visits to AGeFIB field offices in the review process.

Financial management performed well during the first three years of implementation. In late 2009, performance weakened due to the departure of AGeFIB principal accountant, which caused delays in disbursement requests and difficulties managing disbursement categories (ICR, p. 9). The response was to intensify internal audit functions and recruit additional personnel. Performance weakened again towards the end of the project, due to weak management of the recovery of the special accounts, which caused cash flow constraints and delays in implementation. The ICR (p. 22) reports minor delays and shortcomings in audits, which were not serious and promptly corrected.

Procurement functions were performed satisfactorily overall. A procurement plan was developed during project preparation, and training in procurement was delivered to the communities as part of the Grassroots Management Training. After a period of satisfactory performance until mid-2010, the ICR reports issues related to significant delays in obtaining approvals from the National Public Contracting Office as well as delays due to financial constraints and turnover in personnel at AGeFIB (see above).

c. Unintended Impacts (positive or negative):
None noted.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Negligible to Low
Negligible to Low
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

Lessons 1 is derived from the ICR (re-interpreted by IEG). IEG offers Lessons 2 and 3:

1. The community-based development approach does not work "automatically" simply because poor communities are engaged in their own development, but instead requires a number of conditions to work effectively, namely:
- it needs to be structurally and explicitly integrated into decentralization policies. The project placed much emphasis on the need to have the correct legal and regulatory framework in place to ensure strong and sustained commitment of the government to decentralization. The project team insisted, for example, that the Law 2001-07 on decentralization had to be passed and enforced as a pre-condition of effectiveness. As a result, the legal framework of Benin recognizes the ability of the community level to engage in development within the growing level of responsibility and action of the commune government, which creates real opportunities to have more resources and decisions at the local level;
- it needs to be supported by strong and relevant capacity building. The project delivered training at all levels: national, sectoral, and in particular at the commune and community level. The Grassroots Management Training program delivered by the project included modules that had been piloted with success elsewhere in Africa. The results of the ex-post evaluation indicate that the training led to a significant gain in knowledge of how to implement development sub-projects;
- it needs to be integrated in the broader macro-level dialogue, encompassing fiscal decentralization and sectoral deconcentration. Community-based development requires not only decentralization in decision-making, but also fiscal decentralization and deconcentrated spending of sectoral ministries.

2. Investing in analytical studies and M&E tools is essential to 'adjust' implementation, document results, and eventually 'back up' a specific approach. The project invested considerably in M&E, although more to track target indicators than to adjust and correct the course of implementation. To some extent the difficulty of using M&E to correct implementation in "real time" reflects objective limits in local capacity, which this project tried to address with community training. On the other hand, the project did commission an impressive number of studies, some of which were very effective in demonstrating the cost-effectiveness of the community-based approach in providing social infrastructure, in particular schools. These types of analytical studies are quite powerful in supporting and "backing up" a specific approach.

3. All the components of a community-driven development project should be coherently integrated and receive adequate support to achieve best results. The project proved quite effective in improving access to basic services through improved infrastructure; however, it performed less well in delivering on the financial and the income-generating activity component. The ICR does not provide a clear and specific reason why the discrepancy in performance between these two components occurred, but it indicates that other parts of the project received more attention and were prioritized over the financial and (especially) income-generating activities component when the project experienced some uncertainties about funding. The financial and income-generating activity component received also a much smaller budget allocation at design. It also experienced delays on several fronts from an early stage (ICR, p. 16) -- one may assume for inadequate preparation and/or technical/managerial skills. The project team, however, indicated that those sub-projects that were completed under the income-generating activity sub-component showed in many cases promising results at closing.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is very complete and well organized. It provides detailed and comprehensive Annexes summarizing the main findings of the studies commissioned. The ICR documents outcomes clearly, using the results of two Beneficiary Assessments, technical audits, and efficiency studies. The number and quality of M&E tools that the ICR team used to derive information on outcomes and outputs are satisfactory, and their methodology is well explained in the main text and annexes. The lessons learned found in the ICR are a bit generic and would have benefited from more details and specific examples, but they are coherent with the analysis and discussion reported in the ICR. The ICR is candid in recognizing some shortcomings of the project, in particular its under-performance with respect to the income-generating activities sub-component. The ICR clearly reports all the indicators of underperformance and discusses some of the potential reasons.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions