Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Education & Training Support


  
1. Project Data:   
ICR Review Date Posted:
09/23/2013   
Country:
Cote d'Ivoire
PROJ ID:
P035655
Appraisal
Actual
Project Name:
Education & Training Support
Project Costs(US $M)
 76.40  60.40
L/C Number:
C3079
Loan/Credit (US $M)
 53.30  60.40
Sector Board:
Education
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  05/29/98
 
 
Closing Date
12/31/2002 08/31/2012
Sector(s):
Primary education (39%), Secondary education (38%), Tertiary education (12%), Central government administration (6%), Sub-national government administration (5%)
Theme(s):
Education for all (25% - P) Gender (25% - P) Rural services and infrastructure (24% - P) Participation and civic engagement (13% - S) Decentralization (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Susan Ann Caceres
Judyth L. Twigg Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:


    According to the Development Credit Agreement (DCA, Schedule 2), the objectives of the project were "to assist the Borrower in its efforts to strengthen its capacity to carry out the comprehensive basic education reform program; increase access to, and improve completion rates in, primary education; and improve the quality of instruction and learning outcomes for students in primary and lower secondary education." The Project Appraisal Document (PAD, p. 2) stated that the objectives were "to support the sectoral strategy set out in the National Education and Training Development Plan, for which one of the objectives is to achieve universal primary education by 2010. It seeks to achieve improvement in sector efficiency and school effectiveness, and to increase access to basic educational services in targeted under-served areas."

    According to the Restructuring Paper (p. 12) the project objectives were revised "to assist the Ministry of National Education to resume the delivery of education services in primary schools, which implies (1) providing quality school inputs to guarantee teaching and learning in classrooms, and (2) improving access to education and retention of primary school children."

    The Guidance Note (paragraph 4 Resolution of Differences) given operational staff with respect to packages submitted to the Board states that “if there are differences of views between the lawyer, other reviewers, and/or the Task Team Leader as to the consistency of the project documents with the negotiated legal agreement(s), the negotiated legal agreements(s) prevail absent a reopening of negotiations to address the discrepancy.“ In this review, the objectives noted in the DCA, as well as the revised objectives, will be assessed over the entire project period, with the ratings weighted according to the percentage of the credit disbursed at the time of restructuring.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 08/15/2008

c. Components:

There were five components (the actual costs listed reflect disbursements between 1998 and 2004):

Improving Learning at the Primary and Secondary Levels (appraisal, US$18.00 million; actual, US$2.50 million) contained five sub-components:

    • Improving curriculum content and implementation and testing of the reading and mathematics programs was to help teachers to better evaluate their students' learning. The sub-component was to support the reform of the Ecole Normale Superieure. Teachers were to be trained to develop individualized learning programs for all students considered at risk of failing the school year. Remedial courses were to be organized.
    • School-based pre-service and in-service teacher training was to reduce the length of pre-service training to two years, while in-service training was increased. At the secondary level, technical support would be given to the Secondary Teacher Training Institutes to make teachers at the colleges more versatile to strengthen mathematics and science teaching, and to develop new minimum competency levels for secondary students. Links were to be established with the ecoles and colleges to respond to their training needs and take advantage of innovations generated by these establishments. A distance training program for teachers who did not have their high school teaching diploma was to be developed.
    • Applied research and pedagogical innovations was to build on the demonstration school ecoles temoins which helped teachers, school directors, and communities focus on the process of improving classroom teaching and school operations. A network of 750 primary schools and 180 colleges was to be established. A small program of school-based initiatives was to be piloted to improve the quality of schools, particularly for girls.
    • Provision of textbooks in the targeted areas was to improve the availability and use of textbooks in primary and lower secondary schools in poor areas. At the primary level, this sub-component was to provide a set of 2 books per student (French and math) for grades 1 and 2; a set of 4 books per child (French, math, science, and history/geography) for grades 3-6; and guides for each teacher. Books were to be distributed to schools and rented to students for CFAF 250 per set per year. All students would have access to books regardless of parents' ability to pay. The schools were to use the funds collected to finance the operating costs of the rental scheme. Similarly, there was to be provision of textbooks for lower secondary level students, where the books would be rented to students for CFAF 3,000 per set per year. In the schools not targeted by the project, textbooks would continue to be sold, but at a 6% reduction in 1997/1998 prices. Teacher training on preservation of textbooks was to be provided.
    • Student learning evaluation system was to update a previously developed curriculum-based test and be administered twice a year. The institutional arrangements for the creation of testing and evaluation capacity were to be defined during the first year of the project.

Expanding Access to Primary Schools and Colleges (appraisal, US$19.50 million; actual, US$0.90 million) had two sub-components.
    • School construction and renovation was to create 900 new classrooms and 8 colleges, and rehabilitate and equip 300 classrooms.
    • School maintenance was to provide technical support to municipalities and communities to build their own primary schools and maintain school infrastructures. Technical and financial support was to be provided to establish standards and procedures for the rehabilitation of schools, a country-wide maintenance program, school model plans, and monitoring of construction and renovation of community-built schools; and community awareness programs, maintenance manuals, and other technical documents.

Developing Institutional Capacity (appraisal, US$6.10 million; actual, US$0.25 million) included five sub-components:
    • Decentralization was to support the Ministry of Education and Basic Training in the coordination of phasing of its decentralization action plan to improve efficiency of basic education management. It was to provide resources for reorganization, redeployment, and training, computerization of the Ministry of Education and Basic Training, and equipment needed for supervision.
    • Strategic planning and management was to establish a management information system for primary and secondary school records; establish school mapping with decentralized structures; strengthen the observatory's monitoring capacity; establish regional monitoring and evaluation; and increase capacity to monitor girls' schooling.
    • Improved resource management was to support budgetary and financial management at the central and decentralized levels; improve the quality of personnel records and decentralized recruitment of teachers by budgetary posts; promote parent participation in basic education management; and monitor and evaluate existing agreements with private schools.
    • Creation of a Girls' and Women's Education Promotion Unit was to monitor girls' education activities such as a database on number of girls; legislation protecting young, school-age girls in school; studies concerning girls' education; and support for NGOs and other associations.
    • School Health and Nutrition was to support pilot activities to promote the health and nutrition of school children with community participation; build outreach capacity for the Directorate of Extra-Curricular and School-Fund Activities; and monitor the Fichier Central de l'Education Nationale at the Planning and Statistics Directorate of the Ministry of National Education and Basic Training.

Sector Coordination and Strategic Planning (appraisal, US$ 7.50 million; actual, US$0.40 million) included three sub-components:
    • Strategic planning and management of higher education was to finance improvements in the management information system of academic records; capacity building in planning and management of the sub-sector; strengthened capacity to monitor and accredit private institutions; strengthened human and financial resource management at the university and Financial Affairs Directorate/Ministry of Higher Education, Research and Technological Innovations levels; and improved strategic programs such as teacher training and science and technology education and training.
    • Strategic planning and management of vocational and professional training was to help strengthen the management capacity of the recently established Ministry of Technical Education and Vocational and Craft Training by supporting the implementation of a management information system; strengthening management capacity; improving the regulatory framework to increase school autonomy and strengthen capacity to pilot and evaluate signed agreements with private sector; and introducing alternative methods of professional training in coordination with a project financed by GTZ.
    • Sector coordination and dialogue was to support an electronic communication and information system between the three ministries; and sector coordination and monitoring of pre-investment studies, economic analysis work, and private sector development.

Project Management and Coordination (appraisal, US$ 1.50 million; actual, US$0.80 million) was to finance central services based in Abidjan and regional branches (installation, contractual personnel costs, and operating expenses), and the operating expenses of the Project Implementation Bureau of the Ministry of National Education and Basic Training (installation and contractual personnel costs linked to the project).

The restructuring modified the project activities to focus almost entirely on supporting the restoration of basic education service delivery. After restructuring, there were four components (the actual costs listed reflect disbursements between 2008 and 2012):

Improving the Quality of Basic Education (appraisal, US$27.70 million; actual, US$25.70 million) was to develop, print and distribute textbooks and teachers' manuals to achieve a ratio of three textbooks for each student in grades one and two and five textbooks for each student in grades three to six; finalize the primary curriculum and train teachers and supervisors; strengthen the national student evaluation system and conduct evaluations in one primary and secondary grade, as well as analyze and distribute the results; rehabilitate teacher training centers and provide materials to student teachers; provide literacy training for 3,000 people, of which 75% were women in the zones under Forces Nouvelles control; and provide food to 400 primary schools in the regions of Abidjan, Yamoussoukro, Dimbokro, Daloa, and Gagnoa through the World Food Program.

Supporting Access to Primary and Lower Secondary Education (appraisal, US$16.20 million; actual, US$15.90 million) was to construct and equip 300 new classrooms and 8 lower secondary schools.

Developing Institutional Capacity (appraisal, US$17.80 million; actual, US$14.90 million) was to support capacity building for each of the three Ministries of Education. It was to set up and staff a 4-person strategic executive unit located in the Minister's Cabinet; develop and install an education and management information system; and strengthen human resources in charge of education statistics, human resource management, and school mapping. It was to equip the central and decentralized Ministry of National Education. For the Ministry of Technical Education and Vocational Training and the Ministry of Higher Education and Scientific Research, this component was to develop and install a management information system; provide technical assistance in areas of statistics, human resource management, and studies for the elaboration of national strategies in technical and vocational training sector and higher education; and provide vehicles and equipment for planning divisions and some decentralized structures in the Ministries.

Project Management (appraisal, US$4.40 million; actual, US$3.90 million) was to finance the costs of the project execution unit, which was responsible for fiduciary management, civil works monitoring, and evaluation, as well as completion of reports.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Costs: During the four-year period of suspension (see Dates), there were positive exchange rate gains from the original amount (SDR 39.6 million). This meant that the project appraised for US$53.3 million in 1998 was equivalent to US$66.1 million by July 2008. During restructuring, resources were shifted from expanding access (component 2) to improving quality (component 1), institutional strengthening (component 3), and project management (component 4).

Financing: The project was supported by an IDA credit.

Borrower Contribution: Borrower contributions were planned to be US$23.10 million; however, during a country portfolio restructuring, the Borrower contributions were eliminated. The Government notes that it contributed 3,648 billion CFA Francs (ICR, p. 47).

Dates: The project became effective on September 25, 1998, which was a politically volatile time in the country. In 1999 the government went into arrears, and the project was suspended on October 31, 2000. By January 30, 2002 the government cleared the arrears and implementation resumed. On December 23, 2002 the project was extended by six months from the planned closing date of December 31, 2002 to June 30, 2003 because of civil conflict. In June 2003 the country portfolio was restructured and the project closing date was extended from June 30, 2003 to June 30, 2004. On June 14, 2004 the government went into arrears again, and project disbursements were suspended on June 15, 2004. As the turmoil escalated into a full-scale civil conflict between 2002 and in 2004 the project was suspended, along with all other Bank activities.

After a Peace Accord that was signed on March 4, 2007, the Bank provided a pre-arrears clearance grant and reactivated the country portfolio, which included resumption of disbursements on April 2, 2008. The project resumed on July 29, 2008 after a country portfolio restructuring, which extended the closing date from June 30, 2004 to August 30, 2010. Given the extended conflict and resulting destruction to the education sector, the project was restructured on August 15, 2008 to assist with recovery needs and respond to post-conflict priorities. In 2009 the project was restructured to increase procurement thresholds. On April 5, 2010 the project closing date was extended from August 30, 2010 to August 30, 2011. After the October 2010 elections, the country had another phase of civil conflict, and in February 2011 disbursements were suspended until June 7, 2011, when the conflict ended and the government cleared its arrears. On August 16, 2011 the closing date was extended from August 30, 2011 to August 30, 2012 to allow time to complete activities to achieve the development objective. The project closed on August 31, 2012.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Original Objectives: Substantial

In 1997, the net enrollment rate at the primary school level in Cote d'Ivoire was less than 51 percent (41 percent for females). Access to education for children in poor households was 41 percent and 30 percent amongst the poorest. Quality of education was poor, with 22.8 percent repetition of first grade and 42.5 percent of sixth grade, and the primary school completion rate was less than 40 percent. The primary gross enrollment rate (GER) was 80.9 percent for boys and 61.4 percent for girls in 1996/1997. The original objectives were therefore responsive to country conditions at the time of appraisal.

The objectives were also aligned with several government policy documents at appraisal, including the Consultative Group Sector-wide reform framework, the Education Strategy and law approved by the National Assembly, and the National Policy Statement on Education and Training. The objectives are also relevant to the current National Education and Training Development Plan and Heavily Indebted Poor Countries Initiative agreement and education sector targets. The objectives are aligned with the current Bank Country Assistance Strategy (FY10-FY13), which focuses on growth through rapid poverty reduction and improved public sector management, and recognizes the strong links between universal basic education, economic growth, and poverty reduction. The Strategy (p. 5) notes that the country is unlikely to meet the Millennium Development Goals (MDGs) related to primary education, and so it emphasizes improved access to basic education services.

The project objectives were relevant to the country's sector needs when the project was developed, and continued to be substantially relevant when the project closed. However, the objectives emphasized a wide range of aspects of the education system that needed improvement, while the current Country Assistance Strategy narrowed its focus to access, given the changing context of the country are a result of years of civil war.

Revised Objectives: High

Given the impact of the civil conflict, gross enrollment rates dropped from 76 percent in 2001 to 53.3 percent in 2002/2003. During the conflict, there was significant destruction of classrooms, deterioration in availability of textbooks, and closing of some teacher training institutes. The new objectives were aligned with the government- and donor-endorsed Education Recovery Restoration Plan that was prepared in mid-2007. This plan focused almost entirely on supporting the restoration of basic education service delivery. The revised objectives were highly relevant when developed and continued to be highly relevant to the end of the project, given that the current Bank Country Assistance Strategy (FY10-FY13) recognizes the strong links between universal basic education, economic growth, and poverty reduction. The Strategy (p. 5) notes that the country is unlikely to meet the MDGs related to primary education, and so it emphasizes improved access to basic education services.

b. Relevance of Design:

Original Design: Modest
Design included activities to assist the government in reaching its goal of universal basic education and improved quality by expanding schools in underserved areas, focusing on girls, building sector capacity to improve the delivery of education services and training of teachers, and providing textbooks and learning assessments. There were activities to continue sector-wide dialogue in the areas of vocational training and higher education. Girls were integrated into each component, since there were significant gender imbalances in education. However, there were shortcomings. Overall, project design was insufficiently detailed and left too much to be determined during implementation. For example, there was little analysis of planning to demonstrate that the textbook rental scheme would be sustainable and not detrimental to the poor. Design was too complex, particularly considering limited capacity in the Ministries.

Revised Design: Substantial
The revised design supported stabilization after the crisis and assisted the government in addressing the Peace Accords with the provision of basic social services. The scope of the project was focused to support the restoration of basic education service delivery and use additional resources to support teacher training and literacy and respond to the country's food crisis. The revised design simplified the components to focus on the basic education activities that directly addressed post-conflict priorities. While girls were not specifically addressed, they benefitted from the rehabilitating of the education system. There was a logical and plausible relationship between the planned activities, anticipated outcomes, and achievement of objectives.


4. Achievement of Objectives (Efficacy) :

During the course of the project, there were several exogenous factors (civil war, suspension of project activities, food crisis) that impacted project implementation and achievement of objectives. The evidence noted below mainly relates to the post-restructuring period of the project. The ICR (p. 16) indicates that progress toward achievement of the objectives prior to restructuring was mixed, varying between satisfactory and unsatisfactory progress depending on the political situation and the Bank's and government's access to information to measure progress.

Strengthen capacity to carry out the basic comprehensive education reform program (Original Objective): Modest

Outputs:
The project supported the three ministries at both the central and lower levels with critical office equipment, 212 vehicles, and 8 motorcycles.

The Ministry's capacity was reinforced by the project through training. As evidence of the improved strategic and operational capacity of the Ministry of Higher Education and Scientific Research, the ICR indicates that a sector strategy was developed and is now being used to guide higher education reform.

The Ministry of National Education developed an Education Sector Plan that was endorsed by the Global Partnership for Education.

The project established an inter-ministerial Task Force. It produced a Medium-Term Expenditure Framework, sector policy, and statistical yearbooks as well as assessments that were used to inform preparation of the new Global Partnership Project.

An education statistics production system was established, while the Education Management Information System was not developed.

Increase access to and improve completion rates in primary education (Original Objective): Modest

Outputs:
9 lower secondary schools were constructed, which exceeded the target (8 schools). One additional school was rehabilitated. Schools were located in rural areas and/or to relieve transition pressure from grade 6 to lower secondary or reduce overcrowding. This was expected to provide an increase in capacity by 5,000 students. The ICR does not state the actual number of students enrolled.

300 classrooms were built or rehabilitated at the primary level, which met the target (300). The ICR (p. iv) indicates that an audit showed the structures were sound and of good quality. This construction was expected to provide an increase in capacity of 13,800 students. The ICR does not state the actual number of students enrolled.

30,870 student desks were procured, which exceeded the target (25,000). The increase in the number of student desks was needed to respond to the shortage following the crisis, during which many desks were used as fire wood. The total number of desks acquired over the life of the project was 38,351. The project also provided teacher desks, chairs, and bookcases to 93 schools. In terms of furniture, an analysis in 2009 showed that 51.89% of students in grade 2 and 63.07% of students in grade 5 shared a desk with only one other student. There was a significant and negative relationship between the number of students per desk (if greater than or equal to three) and student learning (ICR, p. 18).

400 canteens were supplied with food between 2008 and 2010. The project helped to re-stock the World Food Program's storage facility after it had been pillaged, to avoid a food crisis. The ICR notes that a 2009 study found a positive relationship between schools with canteens, electricity and latrines and student results on national examinations.

Outcomes:

The primary completion rate increased from 46% in 2008 to 64.2% in 2012, which exceeded the target (50%).

The gross primary enrollment rate increased from 74% in 2008 to 83.6% in 2012, which exceeded the target (82%).

The gross enrollment rate in first grade increased from 71% in 2008 to 76% in 2012, which missed the target (80%). The trend is positive and progress is substantial considering the short period of time after the civil war.

The primary ratio of girls to boys was 45% in 2012, which did not meet the target (48%). No baseline was provided and so the extent of change is not determinable.

The secondary ratio of girls to boys was 41.9% in 2012, which met the target (40%). No baseline was provided and so the extent of change is not determinable.


Improve the quality of instruction and learning outcomes for students in primary and lower secondary education (Original Objective): Modest, based on lack of evidence of learning outcomes.

Outputs:
During the first phase of the project (by January 2000), 1.16 million textbooks were provided, although certain schools did not receive the full set of textbooks needed. During the second phase, 6,791,971 textbooks were purchased, which exceeded the target (6,326,215). This distribution target assisted the government in meeting the Heavily Indebted Poor Countries Initiative target for textbooks per child.

13,665 pre-service teachers were trained so that they had minimum qualifications and certification, which exceeded the target (7,276).

Primary curriculum reform was implemented (competency-based curriculum). During the first phase of the project, no significant progress was made in terms of school-based pre- and in-service teacher training. Draft legal texts were prepared to bring about teacher training reforms (lower number of years of training). Study tours to Guinea and Senegal by a consultant led to a draft operation manual for the ecoles temoins, and training of regional coordinators and analysis of existing models.

14 Teacher Training Institutes were re-opened and 2 of them were fully renovated, which exceeded the target (4 institutions in rebel controlled zones re-opened). Pedagogical materials for staff and students were provided for the 14 institutions. At the secondary level, the government has trained 3,897 additional teachers since the start of the 2008/09 year, and an additional 3,070 teachers enrolled in 2012.

66 local School Management Committee Coordinators were trained. A technical audit was done of 3,000 School Management Committees to identify constraints to inefficient functioning, and the lessons were used in the development of the follow-up Global Partnership for Education project.

Information and sensitization campaigns about the schools were carried out. Since roles and responsibilities of the institutional structures could not be defined, no progress was made.

9,790 teachers were provided in-service training, which met the target (9,790). 2,400 teachers were trained in a competency-based approach, and 7,390 teachers received upgradings.

A preliminary minimum exit profile for students graduating from elementary and lower secondary school (grades 6 and 10) was prepared.

Assessments of student achievement outcomes were carried out in Grades 2, 3 and 5, which exceeded the target (assessment in one grade in primary and one grade in secondary). Baselines were established for each of the grades. A follow-up assessment was to be done 2012/2013 for Grades 2, 3, and 5. The secondary assessment was not implemented (but instead two primary grades were added to the assessments that were carried out) due to the 2011 crisis that caused delays in its development. The resources that were allocated for the secondary assessment were used to conduct a technical audit of the Teacher Training Institutes, since teachers were completing the program without appropriate qualifications. The data from the audit were used to redesign the training programs in the follow-up Global Partnership for Education project.

Outcomes:

The repetition rate decreased from 22% in 2008 to 18.7% in 2012, which missed the target (14%). However, the target is high for the context/circumstances. The ICR also reports that there are disparities between the individual grades. The Grade 1 rate decreased to 14%, while the Grade 6 rate was 31% because students could not transition to lower secondary, as there were limited spaces. When the upper grades were removed from the calculation, the rate dropped to 17% (ICR, p. iii).

The textbook-to-pupil ratio was 1:3 in primary and 1:5 in secondary, which met the targets (1:3 primary and 1:5 secondary). No baseline was provided, thus the extent of change is unknown. The ICR (p. 18) reports that the grade 6 primary exam success rate was higher for students who had reading and mathematics textbooks than the students in schools without such books. The ICR also reports that having two or more students share a mathematics book significantly and negatively impacted their ability to learn. Analysis showed that having a reading textbook per student could increase a student's success rate by 0.5 percentage points, and by 1.5 percentage points in the case of mathematics textbooks.

Resume the delivery of education services in primary schools (Revised Objective): Substantial, as this objective aimed at the more modest goal of putting services back in place after the war.

Other outputs:
3,000 adults participated in short-term literacy training, which met the target (3,000 of which 75% were women). 120 organizers from three NGOs provided the literacy training.


5. Efficiency:


Project efficiency is difficult to measure, given the exogenous factors that impacted project implementation, but the project was able to provide value for money.

Because the project was in the country portfolio after the civil war ended, the Bank was able to activate the project and respond to the needs of the education sector in a much quicker time and in a less costly manner than with the preparation of an emergency project. It took less than one month in the case of all three suspensions to re-engage and resume project implementation in support of education service delivery, and average project preparation cost is US$ 611,000 in the Africa region. Efficient project delivery is needed in a post-conflict setting, where early results can create momentum for longer term institutional transformation (World Development Report 2011).

While average costs for construction of classrooms were higher (but not excessively higher) than originally estimated, with an average unit classroom cost of US$21,118, there are several reasons for this. First, the original estimates were calculated in 1998 and did not consider inflation ten years later. Second, during the second phase of the project (after the civil war), there was limited capacity within the country for school construction. While costs were higher than under similar projects in Benin (US$12,931 to US$20,000 ) and Burundi (US$24,343), they were not excessively high and showed cost-efficiency against comparators within Cote d' Ivoire. For example, the unit cost of classrooms built outside the project in Cote d' Ivoire ranged from US$21,325 to US$41,408.

The project was intended to shift textbook publishing to the private sector to increase availability and cost savings. The Government was reluctant to lift the monopoly on the production of primary school textbooks during the first phase of the project, and so little progress was made with the opening up of textbook development and production. While 1.16 million textbooks were provided, not all schools received the full set during the first phase of the project. During the second phase of the project, the project provided 6,791,971 textbooks with a unit price of US$1.30, which is lower than the original projected unit cost (US$2.40). The actual unit cost is similar to costs realized in Burundi (US$1.10) and Niger (US$1.40).

The ICR also reported spillover effects from the project's capacity strengthening, with the establishment of a task force under the Ministry of National Education and Basic Training. The proven capacity of this inter-ministerial body encouraged other donors efficiently to use it to help coordinate their investments in the education sector: Japan, Germany, and France.

Efficiency: Substantial

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Original objectives: Based on Substantial relevance of objectives and Modest relevance of design, Substantial efficiency, and Modest achievement of all three objectives, the outcome based on the original project objectives is rated Moderately Unsatisfactory (corresponding to a rating of 3 on the 6-point scale).

Revised objectives: Based on High relevance of objectives, Substantial relevance of design, Substantial efficiency, and Substantial achievement of the one objective, the outcome based on the revised project objectives is rated Satisfactory (corresponding to a rating of 5 on the 6-point scale).

The overall project outcome is based on the average of these ratings, weighted by the amount of the total loan disbursed before (10.4%) and after (89.6%) restructuring, as follows:

Outcome, original objectives = Moderately Unsatisfactory (3) x .104 = 0.312
Outcome, revised objectives = Satisfactory (5) x .896 = 4.480

Total: 4.792, which rounds to 5, or Satisfactory

The revised design and activities addressed the key needs of the sector and eliminated activities to facilitate implementation.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

Given the post-conflict status of the country, there are risks to the development outcome (fragile political situation, weak institutional capacity, and limited resources). However, there are several factors that help to mitigate some of the risks. A successful inter-ministerial Task Force was established that continues to play a key role in coordination, collaboration, and capacity-building. Members of the Task Force continue to receive training related to financial management and procurement. There is a Global Partnership for Education project currently being implemented, which directly supports and sustains the achievements of this project.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

The Bank team considered implementing a sector-wide program, but did not given that strategies for the sub-sectors of technical and higher education were not developed. Preparation included dialogue with the three education Ministries (Ministry of National Education and Basic Training, Ministry of Technical Education and Vocational and Craft Training, and Ministry of Higher Education, Research and Technological Innovation) and other donors to develop sustained political leadership and ownership. The National Education and Training Development Plan was used to build consensus to support the education reforms. Preparation was collaborative and learned lessons from previous operations, such as the need for specific interventions in the education sector rather than reliance on a programmatic approach through the Ministry of Finance, and ensuring that the project was responsive to national and regional economic conditions and demands for skills. Risks were appropriately identified and mitigated; however, country risks were not, as this was not common practice at the time. Design was reviewed by peer reviewers, which was the process at the time to provide guidance. Implementation arrangements were developed that included all three education Ministries to build ownership amongst the entities. Areas of responsibility were defined that were reflective of the Ministries' capacity and previous experience with Bank projects.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:

During the first phase of the project, an education economist and procurement operation staff were based in Abidjan. The country presence of Bank staff facilitated dialogue with the government. During the period of suspension, the Bank team kept dialogue open. After restructuring, institutional arrangements were not changed, since the Bank team judged them to be adequate and valid. Given the impact of the civil conflict, the Bank refocused the project to address restoration, restructured the project, and revised the objectives, and also simplified design to accommodate the capacity constraints exhibited within the Ministry of National Education and Basic Training during the first period of implementation. The Bank team worked quickly to restructure the project so that resources could be used rapidly by the Ministry to re-establish the education system. The Bank used ratings to signal deficiencies in performance and acknowledge improvements. The Bank team provided support to overcome capacity issues and helped find technical assistance to support implementation. When Bank missions were suspended (due to the unrest), the team maintained supervision through video conferences and filed supervision reports. The Bank team reclassified the project's Environmental category (and the appropriate documents were prepared), to account for changes in Bank operational polices. The TTL and financial management and procurement specialists were placed in Cote d' Ivoire in 2009 to be able to work more closely with the government to solve project problems quickly and support implementation. The Bank team arranged for various extensions, which provided the time for the project to achieve its objectives during an important period in the country's history, when resources were particularly needed.

Quality of Supervision Rating: Highly Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

During the early stages of implementation counterpart funds were provided as agreed. During the first phase of the project, the Government's performance was uneven largely related to the political turmoil and civil conflict. The Government fell into arrears four times, and the civil conflict caused the Bank portfolio to be suspended for four years. The Government was reluctant to lift the monopoly on the production of primary school textbooks as well as to support schools in the North because they were not under central government control. This delayed textbook production and classroom construction. Progress in the implementation of Component 3 (Sector Development and Coordination) slowed due to changes in ministers and subsequent turnover of key staff. The inter-ministerial committee did not function as intended. The introduction of a complex Public Finance Management System hampered project management, as it required multiple approval steps and delayed disbursements and payments to contractors. After the 1999 coup, the government's budget was delayed in 2000 and the Government maintained its contributions, but it went into arrears with the Bank. The arrears were subsequently cleared in 2002.

The Government remained accessible and open to dialogue even during the period of suspension. After restructuring, the Government showed strong commitment to the project, and its performance was steady. It implemented several of the Bank's recommendations (needs assessment, preparation of a new Country Status Report and Medium-Term Expenditure Framework, and learning assessments for Grades 2, 3, and 5). With the information gained from these reports, the Government was able to focus its support where most needed.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

The Ministry of National Education and Basic Training coordinated all communication with the Bank, while the Ministry's Project Implementation Bureau oversaw financial management and procurement activities. An oversight group composed of representatives from all three education ministries was developed.

During the initial period of time after effectiveness, good progress was initially made but then slowed (see Section 9b). Despite the various factors impacting progress, the implementing agencies continued to implement project activities with counterpart funds and attempted to meet supervision and reporting obligations when the country situation permitted.

During the second phase of project implementation (2008-2012), steady progress was made. As the Ministry's Project Implementation Bureau re-engaged, project disbursements quickly moved (in a two-year period of time, they increased from 10.4 percent to 68.9 percent with 95% committed). After a ten-year period of civil war and unrest, the implementing agencies were able to construct and rehabilitate/equip classrooms, deliver textbooks, and provide management training. Despite the civil unrest after the October 2010 elections, project activities continued. By closing date, all activities were completed, and by project closing 99.2 percent of funds had been disbursed. There was appropriate monitoring of environmental safeguards. The ICR (p. 16) notes that the Ministry of National Education and Basic Training improved capacity to deliver education services, as well as its ability to monitor and evaluate the education system. However, there were some capacity limitations in the Ministry's Implementation Bureau in relation to procurement and financial management activities (see Section 11b).

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The Results Framework was derived from the National Education and Training Development Plan and Heavily Indebted Poor Country (HIPC) education targets, which were reflective of the practices of the time. While most of the indicators were appropriate for the original design, there were a few shortcomings. Outcome measures related to improved access relied on outputs (number of places), rather than showing the changes in enrollment rate (overall and in targeted areas). Several of the indicators were not clear or defined, making them difficult to measure. The PAD did not specify baseline, targets, timelines, or data collection responsibilities. Installation of an Education Management Information System was part the monitoring and evaluation system.

b. M&E Implementation:

During the first phase of implementation, data were collected, but efforts were hampered by the civil conflict. The office of the Ministry of National Education and Basic Training provided Bank missions with data to judge project implementation progress, which was reported in aides memoire and supervision reports. The Education Management Information System was not realized during the first phase of implementation due to the political and fiscal instability of the country, which made its procurement difficult.

During the second phase of implementation, the Results Framework was revised and baselines and targets were updated to reflect country conditions after the civil war (increased need for textbooks, literacy training, and desks). The new indicators were aligned with the objectives. Some of the targets were too ambitious considering the context in the country.

Implementation of monitoring and evaluation improved during the second phase. While the Education Management Information System was being developed, an education statistics system was purchased to allow the Ministry to process data quickly. Regional education offices received information technology equipment and training to facilitate the collection of data for the system, which became operational in 2010. A fully integrated Education Management Information System was not fully developed during the project, but is part of a follow-up Global Partnership for Education project.

a. M&E Utilization:
Data were used by the Bank to assess project implementation.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The project was classified Category C (based on the Bank's 1997 Operational Policy), since construction would comply with national environmental laws and no environmental risks were expected. During the 2008 project restructuring, the environmental category was changed to Category B, because it included school construction. An Environmental Assessment was completed and a Safeguard Data Sheet and Environmental and Social Management Framework were prepared. The post-project environmental audit showed that construction and rehabilitation activities were carried out in accordance with the Framework, and the project complied with safeguard requirements.

b. Fiduciary Compliance:

The Ministry's Project Implementation Bureau was responsible for fiduciary management. Performance was satisfactory; however, during the period of civil unrest, financial management was weak. After 2008, the Bank had new procedures for financial management that required new training and upgrading of skills. The Bank provided the training; however, in July 2012 the Bank downgraded the financial management rating because of lingering capacity constraints. After discussion with the Minister of National Education, two additional accountants were hired for the project, which helped to improve performance and led to 99.2 percent of project funds being disbursed by closing. Difficulties included providing up-to-date financial status reports and preparing and submitting withdrawal applications. The financial management software was updated. By closing, there were no outstanding Financial Management Reports. The project submitted satisfactory and timely quarterly interim reports. The 2011 audit report was qualified, due to missing documentation, and the issues were addressed, thus making no qualifications to audits by the close of the project. The final audit report is being processed and was to be submitted by June 30, 2013.

Procurement complied with Bank policies and procedures. However, the Ministry's Project Implementation Bureau had difficulties resolving issues of non-performing contracts in relation to the production and delivery of student desks. Other difficulties included processing procurement documents and ensuring the distribution of textbooks. As a result, procurement ratings were downgraded in June 2012. With technical support, non-performing contracts were cancelled, new contracts were signed, and delivery was completed to schools before closing.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Satisfactory
Given that most of the disbursements occurred after restructuring, the Satisfactory Outcome rating for the second phase of the project determines the overall project Outcome rating. 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

The ICR (p. 24-25) provides several lessons, which IEG has synthesized to the following:
    • Approved projects should not be closed due to suspensions. The project played a key role in providing needed resources to the education sector each time the country emerged from civil unrest. Emergency projects are an option but take time to prepare, unlike this (existing) project where money was immediately available after each Bank re-engagement. This proved invaluable for getting teachers and children back in school quickly after each crisis and helping the country attain "quick wins" towards stabilization.
    • Oversight bodies with the overall picture of system play an invaluable role in advising sub-sector ministries. The task force established under the project was responsible for helping prepare the 2011 Country Status Report, overall sector policy, ensuring coordination of donor activities, preparing the 2011 Global Partnership Education application, and advising all three education ministries. The task force remains the oversight body for the education sector, which has proven instrumental in setting out a policy framework for the sector and ensuring Government ownership.
    • In post-conflict and fragile states, basing task team leaders (TTLs) in-country is essential for ensuring successful implementation of Bank-supported projects. In this case, the presence of the TTL in the country provided invaluable daily support and monitoring of project implementation. Experience with other Bank-supported projects in post-conflict countries has shown that having the TTL in-country is critical for ensuring timely project implementation.
    • Having a Project Implementation Unit outside the government structure is helpful for project implementation in a post-conflict environment. The benefit of having the Implementing Bureau as a semi-autonomous unit outside the Ministry's structure was important for maintaining continuity and staffing that were committed to the project. The Bureau remained operational because key staff were not Ministry staff and therefore were not subject to political staffing changes. Limited project implementation was maintained, even during the 2004-2008 suspension. It was easy to augment the hiring of consultants to provide support. Therefore, in a post-conflict fragile state situation, consideration should be given to establishing a semi-autonomous Unit with the plan of integrating future Units into the organizational structure of the Ministry.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is concise, well written, and thorough. It provides a candid assessment of implementation weaknesses, particularly related to the Borrower/Implementing Agency. The Annex provides a description of each component by phase of project, which is helpful. The efficacy section mainly discusses the indicators in the Results Framework. The Borrower comments are a useful supplement to the review.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions