Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Education For All-fast Track Initiative Program


  
1. Project Data:   
ICR Review Date Posted:
07/08/2013   
Country:
Benin
PROJ ID:
P110576
Appraisal
Actual
Project Name:
Education For All-fast Track Initiative Program
Project Costs(US $M)
 109.7  177.69
L/C Number:
Loan/Credit (US $M)
 76.10  76.10
Sector Board:
Education
Cofinancing (US $M)
 33.8  85.59
Cofinanciers:
DANIDA French Development Agency, KFW, Netherland
Board Approval Date
  05/23/2007
 
 
Closing Date
06/20/2010 06/15/2012
Sector(s):
Primary education (100%)
Theme(s):
Education for all (100% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Susan Ann Caceres
Alain A. Barbu Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:


    The Legal Agreement did not contain the project's development objectives, instead stated "the project, which is part of the Recipient's Program, consists of the following parts: (1) improving access to education, equity and retention; (2) improving quality of teaching and literacy training: (3) improving management and steering of the educational system (p. 4). The Catalytic Fund Program Document (p.1) indicated that the development objective was "to contribute to expanding equitable access to pre-school and primary education and to supporting the development of literacy programs, while improving quality and retention at primary level as well as the management of the sector.” This review uses the version noted in the program document, since the language is more precise and more evaluable.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

There were three components (costs shown below relate only to the FTI-CF portion):
1. Improving Access, Equity, and Retention (appraisal, US$ 48.0 million; actual, US$ 55.6 million) is to rehabilitate more than 2,000 pre-school and primary classrooms, 45 literacy centers, and 12 alternative education centers. Provide literacy training for girls and women and develop national programs for girls' education and for children with special needs. Support an experimental program for alternative accelerated education for overage children; extend multi-grade teaching and developed new primary school canteens.

2. Improving Quality of Teaching and Literacy Training (appraisal, US$ 148.7 million; actual, US$ 14 million) strengthen the qualifications of teachers and management personnel with training and support to training institutions; introduce national languages into the education system on experimental basis; purchase and distribute textbooks, teacher guides, and school supplies for pre-school and primary school; and provide literacy training to 52,000 adults.

3. Improving Management and Steering of the System (appraisal, US$9.5 million; actual, US$ 5.7 million) improve pedagogical and institutional management through the use of student learning assessments, results-based management systems, and external evaluations of the efficacy of pedagogical interventions. Support sector management capacity building to improve the planning, programming, procurement, financial management, and monitoring and evaluation, and development of an information management system with a school map.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost: According to the project team, total project costs were US$ 177,695,783 with pooled funding and Borrower contributions. Actual cost for component one were higher than planned, and lower for components two and three.

Financing: According to the project team, the EFA-FTI Catalytic Fund provided a US$ 75.1 million grant. Danish International Development Agency (DANIDA), French Development Agency, German Cooperation Agency (KfW), and Netherland expressed interest in pooling their funds with those of the Catalytic Fund for budget support to reduce transaction costs for Government. The project team reported that these contributions were: DANIDA (US$ 25,886,196), French Development Agency (US$16,681,792), KfW (US$20,989,784), and Netherland (US$21,646,582).

Borrower Contribution: No Borrower contributions were anticipated; however, the Government paid US$17,383,404 for construction costs not paid by the project (See section 9b), according to the project team.

Dates: On June 17, 2010 the project was restructured to eliminate and modify activities, as well as change indicators, and extend the closing date from June 20, 2010 to December 31, 2011. On December 19, 2011 another six months was added to the project closing date thus, changing it from December 31, 2011 to June 15, 2012 to allow time to complete activities, as well as revise downward targets for indicators (such as completion rate). The project closed two years later than originally planned.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Objective: High
In 2006 the Government endorsed a Ten-Year Education Plan (Plan Décennal de Développement du Secteur de L’Education – PDDSE), which aimed to: (i) improve service delivery to increase access, equality, retention, and learning achievement in basic education, (ii) promote girls’ education; (iii) strengthen the governance and efficiency of the education system through a better management and redeployment of teaching staff and more efficient financing and management of schools; and (iv) re-orient post primary education and training to better respond to the skill needs of a more diversified economy. Prior to the project gross primary enrollment rate had improved from 90% in 2002 to 93% in 2006, and the percentage of female students increased from 42% to 44.4%. School completion rates had increased from 47% to 65% between 2002-2006 for all students, and 34-54% for girls. The repetition rate had dropped from 21% in 2003/04 to 16% in 2005/06. Enrollment in preschool (public,parochial, and private) was low (3.6% in 2006), and comprised children from wealthy families. National statistics mask the significant disparities, by sex, geography and income level, which relate to the uneven distribution of human and material resources among schools. Another challenge in the system was the low level of learning because of: (i) the use of under- or unqualified teachers; (ii) the deterioration of the student-teacher ratio from 40 in 1992-3 to 47 in 2005-6; (iii) the lack of school inspection and performance monitoring; and (v) poor infrastructure (30.8% of schools made of non-permanent materials). The objectives of this project were highly relevant to the sector needs and aligned with the priorities of the Bank's Country Assistance Strategy (FY09-12, current at closure), which emphasized quality education services to better respond to development needs and reaching the MDGs related to gender equality (p. 10 and 18).

b. Relevance of Design:

Design: Modest
Project design incorporated a common funding mechanism that encouraged donor harmonization. Activities were logical to achieve the objectives related to improving quality, literacy, retention, and management capacity; however they were insufficient to realize improvement in equitable access for pre-school and primary education. Considering the capacity of the government and that it had not previously worked on such a scale in the sector and the short duration of the project, the scope was too broad. The ICR reported that the first restructuring was needed to enable the project to reach its objectives and ensure the completion of the remaining activities, as several activities were modified and dropped, suggesting design was too complex.


4. Achievement of Objectives (Efficacy) :


This project contributed to an overall program in which the Catalytic Fund was one of many donors such as DANIDA, AFD, and KfW. Thus, it is not possible to attribute attainment of objectives solely to the efforts of the Catalytic Fund and Bank. The objective "to contribute to expanding equitable access to pre-school and primary education and to supporting the development of literacy programs, while improving quality and retention at primary level as well as the management of the sector” has been split into the following components:
(a) expand equitable access and retention to pre-school and primary education
(b) support literacy programs
(c) improve quality at primary level
(d) improve management of the sector.

Expand equitable access and retention to pre-school and primary education: Modest

Outputs:
    • 486 latrines were constructed, which met the original target (440) and revised target (324).
    • 2, 613 classrooms built or rehabilitated, which met the original target (2015) and revised target (1926). These classrooms did not create new enrollment capacity, but were more permanent and so helped to improve the learning environment and retention particularly during the rainy season (ICR p. 11-13). The project team reported that 781 classrooms were paid for by the Government (See Section 9b.).
    • 501 canteens received food supplies, which met the upwardly revised target (501). The ICR reported that instead of creating new canteens the project supported a daily nutrition assistance in pre-primary and primary public schools in existing canteens. This benefitted 127, 239 pre-primary and primary students by receiving one meal per day, which was higher than the target of 110,000 students.
    • The ICR (p. 6) reported that the following activities were dropped: a national program for girls education and children with special needs and the experimental program for alternative accelerated education for overage children.

Outcomes:
    • The gross enrollment rate in primary education increased from 93% in 2006 to 119% in 2012, which met the upwardly revised target of 112%. This increase may in part be due to the Government's abolition of school fees (ICR p. 15). It is not clear what project activities contributed to the improvements noted in primary enrolment, given that the civil works did not create new capacity.
    • There was an increase in the percentage of female primary students from 44% in 2006 to 48% in 2012, which met the target of 47%. The gender parity index at the primary level rose from .89 (no date) to .92 in 20111. While, the ICR (p. 12) reports that the building of latrines and provision of school feeding may have encouraged families to send more girls to schools, the ICR (p. 18) also states " without more precise data, it is difficult to pinpoint why the rise occurred and what the project's contribution has been."
    • The preschool gross enrollment rate increased from 4.5% in 2006 to 12.3% in 2012, which met the target of 10%. The ICR does not report enrollment by income group and so it is not clear whether the wealthy were the beneficiaries. It is not clear what project activities were conducted which contributed to the improvements noted in preschool enrolment.
    • Primary completion rate increased from 65% in 2006 to 71.5% in 2012, which met the downwardly revised target of 68%.
    • Girls' primary completion rate increased from 54% in 2006 to 65.8% in 2012, which met the downwardly revised target of 60%. The rate of increase amongst girls (11.8%) nearly doubled the overall change (6%).
    • Repetition rates increased from 7.5% in 2006 to 12% in 2012, which did not meet the target of 10%. The ICR (p. 16) calculated a coefficient of internal efficiency based on the repetition and drop-out rates in the primary cycle. The coefficient indicated that nearly half of the resources (47%) were wasted, which was higher than 2006-2007 where it was one-third (32%). The ICR (p.12) suggested one explanation for the higher repetition rate may be the fact that there are more children entering schools from lower socio-economic groups.
    • While there have been improvements in overall and girls' enrollment, the ICR did not provide evidence that access has become more equitable in relation to geography and income level.

Support literacy programs: Modest
Outputs:
    • 45 literacy centers with two-unit latrines were built.
    • There was an increase in the number of adults provided literacy training from 23, 613 in 2006 to 107, 401 in 2012, which exceeded the target of 52,000 adults. The project financed 35, 181 adults, and the Government financed 73, 522 persons with its own resources.
    • 28,940 girls and women enrolled in national language literacy training program, which nearly met the target of 29,200.
    • The number of girls and women trained was 16,448, which did not meet the target (28, 940).
Outcomes:
    • The pass rate in the literacy program was 67%. No baseline was established.

Improve management of the sector: Modest
Outputs:
    • A computerized framework was established. The environmental atlas of the system was dropped. General census of students and database is available. Given the imminent closing, the activities were reduced from setting up the entire Information Management System to only putting in place a computerized framework.
    • 454 Central Ministry staff were trained.
    • The resources from the State budget devoted to education increased from 19.5% in 2006 to 31.1% in 2010, which met the target of 24.4%. The resources from the education budget devoted to primary increased from 47.5% in 2006 to 49% in 2012, which met the target of 47%. The ICR noted the Ministry needs to better deploy resources to reach results/targets.
    • Annual work plans and an operational manual created, "but management structures and systems put in place remain to be perfected" (ICR p. 14).

Outcomes:
None reported

Improve quality at primary level: Modest
Outputs related to professional development:
    • 46,196 teachers were provided in-service training on competency-based pedagogical approaches, which exceeded the target of 16, 000 teachers.
    • 9,916 community teachers were trained with a 3-year course (CEAP), which narrowly missed the upwardly revised target of 10,000 teachers. This represents training nearly every community teacher. The external evaluation of the efficacy of pedagogical interventions was eliminated and so there is no information on its impact or effectiveness.
    • 35 inspectors received training.
    • 1,213 pre-primary teachers received 10 days of training on discovery learning activities.

Outputs related to improving quality:
    • A system for assessing learning at the primary level was established and used in 2011. The 2009 national student assessment was "cancelled because the supervising directorate was not in a position to manage it" (ICR p. 9).
    • 65.9% of classes were visited by Inspectors and Educational Advisors in 2011 which increased from 6% in 2006, and met the original (60%) and revised target (40%).
    • 8,059 schools received subsidies to purchase learning materials for pre-school and primary children, which exceeded the target of 5,763 schools. The project team reported that all public preschool and primary schools benefited from the subsidies.
    • Corrective measures to the curriculum in the teaching of French, Math, home economics, science, arts, and sports were taken, but the ICR did not describe what was done.
    • The ICR reported that the following activities were dropped: the extension of multi-grade teaching and the use of sub-cycles and the introduction of an experimental basis of national languages into the education system.
Outcomes:
    • The public student-teacher ratio remained consistent from 2006 to 2011 at approximately 47 and 47.9. (Target: 49). This indicator was dropped at restructuring. However, there was an increase from 42% in 2006 to 44.5% in 2011 of schools with student teacher ratio higher than 70 or lower than 40, suggesting no improvement in the distribution of teachers (ICR v. iv). The project team clarified that the ratio improved in 2010 because the Government recruited 3300 teachers and since that time none have been recruited, and the ratio most likely has increased.
    • The public textbook-student ratio in reading and math increased from 1.6 in 2006 to 1.93 in 2011, nearly meeting the target of 2. The 2011-2012 data was not available and the indicator was dropped at restructuring. The Government outside of this project purchased 2 million text books. The ICR reported that the provision of textbooks was dropped from the project.
    • The percentage of qualified teachers increased from 45% in 2006 to 60.1% in 2012 (ICR p. v). No target was established and the indicator was dropped at restructuring.
    • 7,971 out of 9,979 teachers who received the three-year in-service training passed the final exam, which represents a pass rate of 80%.
    • 28% of Grade 2 public school students received passing grades in reading. 18% of Grade 2 public school students received passing grades in mathematics. 12% of Grade 5 public school students received passing grades in reading. 11% of Grade 5 public school students received passing grades in mathematics.
    • The 2011 results continue to point to low levels of learning, as the combined score in Math and French were 40 out of 100 for Grade 2 students and 31.85 out of 100 for Grade 5 students.

5. Efficiency:

The ICR mainly described efficiency in relation to continued inefficiencies in the education sector (e.g. recurrent and non-recurrent expenditures, repetition rate, drop-out rate).

The ICR also discussed project efficiency and value for money in relation to construction, school feeding program and training of teachers. The government reduced the per diem meal rate from US$30 to US$5, which reduced teacher training expenses, given the number of teachers trained (46,196 teachers). 28% of the civil works utilized a community approach, which offered the least costly approach (US$12,931 including furniture). In contrast, average unit costs for classrooms built by the Contract Management Agency between 2008 and 2012 varied from US$13, 539 and US$20,603 excluding the cost for furniture (ICR, p. 29). The average unit costs of classrooms built under three other projects in Benin ranged from US$14,907- US$16,936. While the schools built by this project under the Contract Management Agency were higher cost, they were built much faster (4-5 months) than classrooms constructed by the other projects (1-3 years), which enabled the communities to benefit earlier (ICR p. 30). These permanent classrooms replaced "lean-to's", which is expected to bring savings to the communities as this more solid construction should minimize the need to rebuild. The canteens operated by the project were run by community members, which reduced the operating cost to US$ 0.25 per meal.

Efficiency: Modest

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Relevance of objective was high, while the relevance of design was modest due to the complexity and scale given the capacity of the Government. Efficacy for four objectives was modest, as well as efficiency. The project attained a number of outputs, but there were no outcome measures to demonstrate improvements in equitable access by geography and income level, sector management, education quality or literacy.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

The activities of the project are likely to continue, but there are three risks to the development outcome: government commitment, funding, and government capacity. Government commitment has fluctuated during the course of this project. It has slightly increased expenditures devoted to education as a share of total spending and percentage of GDP, but the current level is below FTI's recommended threshhold of 20%. Donors have increasingly funded non-recurrent expenditures from US$49 million in 1999 to US$100 million in current years with the FTI money. Continuation of funding at this level is dependent upon continued donor interest. Another risk relates to the continued need to reform how the Ministry operates and is managed to become more responsive, effective, and efficient. While the project devoted resources to build the capacity of the Ministry, there are weaknesses in the deployment of resources to reach results/targets, procurement, and coordination in the basic education sub-sector. All of these risks combined pose a moderate risk to the development outcome.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

Relevant counterparts in government worked closely with the Bank and partners to prepare this project. Lessons learned from earlier projects were incorporated such as the use of project management agencies for construction and the adoption of decentralized approaches. A thorough assessment of the Government’s financial management and procurement capacity were carried out, both by Bank entities and the EFA/FTI Catalytic Fund Office. The Bank worked with multiple donors and they agreed to enter into a pooled funding mechanism to finance this program. While risks were correctly identified, the measures were not well planned, realistic, and achievable (ICR p. 7). A shortcoming noted in the ICR (20-21), "delays of more than 12 months, at the outset of an operation, to allow a counterpart to put into place the relevant units, systems, and organizations that are needed is neither practical nor realistic given the project’s initial timeframe" (of three years), and so the ICR suggests that the Bank did not do enough to assist the government in finding corrective measures to overcome many of the preliminary obstacles encountered.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

The Bank tried to assist the government in resolving the procedural difficulties that arose. The Bank solicited the help of Benin’s President’s office to help move the agenda forward. The Bank team provided procurement workshops and training to build capacity, since the entities charged with procurement were not familiar with Bank procedures. Project internal ratings appeared overly optimistic, given the low level of disbursements and slow pace of implementation. Ratings of 'project at risk' were only noted in the 2010- ISR 4 and 5. The Bank team conducted two level two restructuring of the project, which reduced the scope of activities and extended the time to allow as much of the activities to be completed. This was viewed as contributing to the success of the project and reducing the complexity of the design and eliminated aspects believed to be unachievable. No mid-term review was conducted.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

Before the elections, the Government worked closely with donors to prepare the project; however, the new government did not re-engage quickly enough on this project. At the end of the project the Government sought ways to expedite activities. It took the Government more than a year to put into place the relevant units, systems,
and organizations, as specified by the effectiveness conditions. There were four Ministries (and numerous directorates) involved in the project and there was poor coordination among them. Despite training, weak capacity persisted within the Ministry. The management structure that was put in place did not function and so the Ministry of Finance created an ad-hoc taskforce (in conjunction with the Program Management Unit) to oversee the project and bring Ministries/Directorates together regularly, which assisted with the implementation and coordination.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:

While a Program Management Unit was the Implementing Agency, four Ministries also had responsibilities related to administration and execution in the project. The role of the Program Management Unit was to leverage government execution, which was at first misunderstood within the Ministries. Implementation was slow and disbursements were low for the first two years. Part of this can be explained by the fact that construction activities began without first obtaining the necessary non-objections from the Bank. As a result, these civil works were deemed to be out of line with acceptable procurement procedures and so the construction had to be paid from the state budget instead of the project. The ICR reported that the Program Management Unit worked hard to implement the project.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

In the Government's Ten-Year Education Plan (Plan Décennal de Développement du Secteur de L’Education – PDDSE) there was a common set of measurable benchmarks to track progress and harmonize donors’ commitment. This measured the broader implementation of the sector plan. In addition, intermediate indicators related to this project's components was also established, allowing for assessment of the project's contribution. The ICR (p. 5) reported that the M&E contained an excessive number of indicators (14), which made it difficult to monitor progress towards the achievement of the objective. The number of indicators was subsequently reduced to six, which were more focused and aligned with the project. The M&E Framework specified baseline and targets, but data collection arrangements and responsibilities were not identified.

b. M&E Implementation:

There were delays in data collection throughout the project and so data was not up to date. Student assessment was intended to occur in 2009 and 2011, while only the later assessment occurred because "the supervising directorate was not in a position to manage it (ICR p. 9). Thus, the project was only able to provide a baseline of student learning, which was not comparable with previous data. The ICR noted the need for more precise education data composed of individual student, teacher, class, and school-level variables.

a. M&E Utilization:

The ICR did not describe how M&E data were utilized by the Implementing Agency and Ministries, but the project team reported that data were used to dialog with the Government during joint supervision missions and at the joint annual review.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:

The project was classified category "B" for Environmental Assessment (OP 4.01) and Involuntary Resettlement (OP 4.12). Since construction took place in existing schools, there was no involuntary resettlement. The ICR reported that satisfactory compliance was present throughout the project. The Environmental and Social Management Framework was implemented by the Contract Management Agency. During supervision missions the Bank team monitored safeguards and found compliance and utilization of mitigation measures. An environmental audit was done in December 2012. The project team reported the study found that construction and equipment of classrooms had minor environmental impact and it made recommendations for future improvements in the area of reforestation and waste management.

b. Fiduciary Compliance:

Financial arrangements and management were adequate and complied with the Bank's requirements. The entities charged with procurement (education units) were not familiar with the Bank's procedures. Other factors which created difficulty with procurement related to inadequate communication between ministries and staff rotation. All of these resulted in bottlenecks and delay. Training was carred out; however, financial and procurement activities remained problematic" (ICR p. 14). Audits were qualified but these related to the delays in accounting.

c. Unintended Impacts (positive or negative):
None

d. Other:
None



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

The ICR (p. 22-24) provided several lessons, which IEG has synthesized to the following:
    • Realism is required. Capacity challenges were identified, but were not dealt with before project effectiveness, which had obvious implications during the course of implementation. Designs must be simple and achievable, but they must also address the education sector needs, which are often complex and interrelated. Thus, capacity constraints cannot be overlooked in future projects and simplicity of implementation must be sought in the design.
    • It is possible to design M&E within a pooled funding mechanism from many donors and track project interventions and thus attribution and contribution of FTI resources to attainment of the objective. Indicators were developed, which were a subset of the larger SWAP program indicators. This allowed the Bank to commit to pooled funding and still undertake its evaluation obligations.
    • Government ownership of the project fluctuated, which adversely impacted implementation. When the planned management structure did not function, the Ministry of Finance put forth an ad-hoc management team that moved the project forward. Future projects will need to identify the roles and responsibilities and formalize the management unit.
    • Focus on building the capacity of systems, not individuals. Given that civil servants rotate between jobs regularly, ongoing training must be put in place. Putting systems in place that are consistently followed by subsequent staff is the way forward. In this project, FTI procedures were new and took time to learn, but helped to rationalize the execution and accountability in the process.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR provides a candid assessment of implementation weaknesses, particularly related to the Bank and Borrower/Implementing Agency. The ICR assesses the attainment of the project's development objective, and discusses attribution and other factors that could have contributed to progress. There were a few discrepancies in the reporting of indicators in various places (i.e., data sheet, efficacy section and Annex 2), which was subsequently clarified with the project team. The ICR did not describe compliance with Involuntary Resettlement and utilization of M&E, points subsequently clarified with the project team.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions