Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Quality Education For All Project - Phase 2


  
1. Project Data:   
ICR Review Date Posted:
08/22/2013   
Country:
Senegal
PROJ ID:
P089254
Appraisal
Actual
Project Name:
Quality Education For All Project - Phase 2
Project Costs(US $M)
 85.0  98.15
L/C Number:
C4231
Loan/Credit (US $M)
 30.0  31.85
Sector Board:
Education
Cofinancing (US $M)
 4.0  4.0
Cofinanciers:
Japan PHRD
Board Approval Date
  07/05/2006
 
 
Closing Date
10/31/2010 06/30/2012
Sector(s):
Central government administration (32%), Primary education (23%), Secondary education (23%), General education sector (16%), Sub-national government administration (6%)
Theme(s):
Education for the knowledge economy (29% - P) Education for all (29% - P) Gender (14% - S) Public expenditure financial management and procurement (14% - S) Decentralization (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Eluned Schweitzer
Robert Mark Lacey Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:


    This project was the second phase of an Adaptable Program Loan (APL) to support the Government's Ten Year Education and Training Program (2005-2015). No third Phase is as yet in operation. The Program objectives are stated in the Project Appraisal Document (PAD, page 5) as follows:

    " The ten-year education sector program (2005-2015), Programme Dcennal de l'Education et de la Formation or PDEF, inspired the Letter of Development Policy, which has been discussed with and validated by all key stakeholders, and which includes, among others, the following strategic objectives: Universal completion of elementary (6 grades) and improved access and quality at all other levels; eradication of illiteracy and the promotion of local languages; an expanded role for local communities in education; promotion and improved orientation of vocational training towards the labor market; elimination of inequality between economic groups (rich/poor), between sexes, between and within regions, between rural and urban sectors, at all levels of the education system; inclusion of children with disabilities; and promotion of girls' education."

    According to the Development Credit Agreement (DCA, Schedule 1, page 6), the project development objective is "to enhance the quality of the Recipient's education system through the improvement of teaching and learning practices."

    According to the PAD (page 7) the project development objective is " to improve teaching and learning practices in the classroom in order to assist the Government in improving education quality, while maintaining its focus on reaching universal primary completion by 2015. The project would do this by:(a) improving both access and retention by expanding the number of middle school places in the system, improving conditions in existing elementary schools, and by supporting additional literacy courses; (b) improving education quality with a focus on elementary education; and (c) strengthening management at central, deconcentrated and decentralized levels to ensure attainment of PDEF objectives."

    The Review is based on the statement of objectives in the PAD since it is more monitorable.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 11/02/2010

c. Components:

The project had four components:

1. Improving access and improving retention in elementary and middle schools (Cost at Appraisal US$22.3 million, Actual Cost US$24 million).- Construction of middle schools in under-served areas (a total of 28 new Colleges de l'Enseignement Moyen would be built)
- Construction of latrines and water points in elementary schools lacking in such services (a total of 300 latrine blocs with 6 boxes each, and 170 water points will be provided)
- Improve the design of literacy courses and provide these to a total of 50,000 learners, 70 percent of whom will be female.
- Supporting training of school personnel, students, and communities in sanitary awareness, environmental issues, literacy programs and school maintenance strategies.

2. Improving education quality in the elementary classroom (Cost at Appraisal US$51.3 million, Actual Cost US$61.12 million )

- Provision of block grants for quality improvement plans at both deconcentrated (education regions and districts) and school levels
- Provision of pedagogic inputs to schools, in the form of textbooks and other instructional materials, and library and reading materials
- Training of teachers, inspectors, school directors and mentor teachers, with specific attention to the development of training modules and the training of trainers on improving quality in the classroom
- Improve systems of examinations and evaluation of learning achievements; and
- Support for sectoral studies in key related sub-sectors, including vocational training and higher education

3. Strengthening management, monitoring and communication throughout the system (Cost at Appraisal US$3.3 million, Actual Cost, US$ 7.37 million)

- Provide support for results-based management within the Ministry at central and deconcentrated levels
- Strengthen the monitoring and evaluation system, building on a strong data collection mechanism; and build capacity for using results for policy development
- Improve communication across all actors in the system, and poll community views on education via a community scorecard.

4. Operation and maintenance (Cost at Appraisal US$3.1 million, Actual Cost US$5.66 million).

To fund operational, maintenance and management costs of the assets financed by the project.

Restructuring
The project was restructured twice. At the first restructuring, the allocation of funds to Component 1 was left unchanged. Component 2 was increased by US$1.3 million. Component 3 was reduced by US$0.5 million, and Component 4 increased by US$2.6 million (CR Annex 1). There were also some changes to the outcome targets: (i) the target for the primary completion rate in 2009 was reduced from 78% to 65%, (ii) the indicator related to the share of Government budget allocated to education was revised in order to align it with the internationally agreed Education for All (EFA) benchmarking; and (iii) the following targets were dropped: (a) the percentage of primary students who would receive at least the minimum acceptable score in reading, languages, mathematics and science by 2009; (b) 50% of the recurrent education budget would be allocated to primary education in 2009; (c) the transition rate from primary to middle schools would reach 57% in 2009; (d) the success rate on middle school exit examinations would reach 60% by 2009; and (e) middle school teachers would teach on average 20 hours per week by 2009; and (f) instructional hours for education would reach 900 hours by 2009.

At the second restructuring, some minor reallocation of funds between components was made.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project cost. The total project cost in dollar terms was US$98.15 million compared to US$85.60 at appraisal. Nearly all this difference can be explained by exchange rate fluctuations.
Financing. The Credit was denominated in Special Drawing Rights. Due to exchange rate fluctuations, the dollar equivalent of the Credit as approved was exceeded by US$ 60,000, while the Borrower contribution rose by 20% in dollar terms. The project also benefited from a Project Preparation Facility (PPF) which was reimbursed. USAID provided parallel co-financing for rural middle schools. Capacity building components were supported by Japanese Grant funds of US$4.0 million. All project funds were fully disbursed.
Borrower contribution. The Borrower contribution in dollar terms rose from US$51.60 million at appraisal to US$62.3 million at closure.
Dates. The closing date was extended twice. The first extension of 14 months (from October 30 2010 to December 31 2011) was granted at the time of the October 2010 restructuring "in the light of the time needed to implement activities and use the credit resources" (ICR, page 36). A further six months extension was granted in December 2011. The project closed on June 30, 2012, 20 months later than the original date.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

High
The project development objective was relevant at the time of appraisal and remained so at closure. It supported the Government's goal of implementing a ten-year mandatory basic education program. The objectives are relevant to the Country Assistance Strategy (CAS) for 2007-10, and for the CAS dated May, 2010, covering the period until 2013. The CASs both identified the need for the Government to further increase resources to education and health in order to attain the Millennium Development Goals. The objectives are also aligned with Senegal's Poverty Reduction Strategy for 2006-2010.

b. Relevance of Design:

Substantial
The objectives are clearly stated and measurable. There was a logical causal chain between the activities financed by the project and the intended achievement of the objectives. For example, construction of junior secondary schools would contribute to achieving the access and transition goals, the construction of latrines and water points would contribute to gender equity, as girls are more likely to go to schools which have satisfactory and appropriate sanitary arrangements and available water. Pedagogic inputs would be expected to have a favorable impact on the quality of teaching and learning in the classroom, as would training teachers. Administrative changes supported by the project would be expected to contribute to both greater efficiency of service delivery and accountability for quality at the local level, impacting the quality of the education service in the long term. One shortcoming of design was inadequate provision for capacity building at the local level. Although training for central government was appropriately provided for, activities to support regional and local education offices were insufficient.


4. Achievement of Objectives (Efficacy) :

The extent to which the project’s objective was achieved -- to improve teaching and learning practices in the classroom in order to assist the Government in improving education quality, while maintaining its focus on reaching universal primary completion by 2015 –- is rated substantial.

Outputs

    • 28 secondary schools were constructed with places for 2,800 students, with some delays according to the ICR.
    • The project supported upgrades in infrastructure at a number of elementary schools, mostly sanitary blocks and water points. Fewer were constructed than envisaged due to the challenges and costs of reaching remote areas.
    • 1,473 block grants averaging US$3,000 each had been made by 2009 to elementary schools, surpassing the target of 1,000. There were also 86 grants to secondary schools (no target). The grants were used to finance school projects encompassing a variety of activities aimed at enhancing the quality of learning in the beneficiary schools, such as acquisition and elaboration of didactic material, development of reading activities and student competitions, and organization of remedial courses and provision of additional teaching time for students with difficulties. Some school projects involved the acquisition of furniture and improvements to the infrastructure.
    • Literacy courses financed by the project reached 53,250 beneficiaries, 90% of whom were women.
    • The project financed the purchase and distribution of 1,812,000 pedagogical manuals and reading books to improve classroom practices at the primary level. Delays in the acquisition of manuals and a shortage of trained teachers restricted the use of the manuals and negatively affected their impact during the implementation period. At the secondary level, the project supported the acquisition of 200,000 pedagogical manuals and 202,000 mathematics textbooks. The project provided resources to 779 schools to establish 3,116 reading corners in 4 classes. Elementary fourth and fifth grade courses were equipped with 30 books each.
    • 28,162 staff in regional inspection structures were trained in the use of didactic materials, the observation of the use of these materials and other topics.
    • Activities relating to the improvement of the assessment system were completed, although the ICR does not specify what these were. The project supported two rounds of national learning assessment evaluations in 2010 and 2012 for several grades. These were not, however, comparable due to changes in the curricula.
    • Studies in key related sub-sectors, including vocational training and higher education, were undertaken.
    • The project supported the regular publication of annual statistical reports by 2009, and the data unit at the Ministry of Education developed a results-based management tool. Progress was made on the elaboration of a single personnel database and on redefinition of the functions of a decentralized system.
    • Performance management frameworks were developed and implemented in all directorates by 2009 in support of the Government’s efforts to move to results-based performance. The ICR (page 30) reports that by project closure, all Directorates and Inspectorates of the Ministry of Education had signed performance contracts. The purpose is to contribute to a change in culture favoring greater accountability and capacity to implement results-based management at all levels, though this is a longer term goal. The project supported the development of an information system to monitor the implementation of the performance contracts.
    • Other contributions to improved management included: (a) training on Information Technology and management tools; (b) upgrading of the hardware and software used by the Ministry of Education, contributing to better tracking and use of resources; (c) an analysis of Ministry of Education activities to better define roles and responsibilities; and (d) training and staffing needs for regional and district levels of administration were identified, although no activities had been implemented by project closure.
    • The number of hours of instructional time for students was increased from 700 to 806 hours per year (more instructional time is known to be a factor in improving learning outcomes). Although this is a significant increase, the number of hours remained below the target of 900.
    • The project also supported the development of a results-oriented Medium Term Expenditure Framework for the education sector in order to achieve a better link between budget allocations and sector priorities.

Intermediate Outcomes: Improved Access and Retention
The evidence produced by the ICR to measure achievement of the intermediate objectives of improved access and retention consists of national level indicators:
    • With regard to access, the ICR (page 26) states that the intermediate outcome indicators were mostly achieved or surpassed. The gross enrollment rate at the primary level increased from 79 percent in 2005 to 94.7 percent in 2012, slightly below the 96 percent target indicated in the PAD. The gross intake rate in the first year of the primary level increased to 106 percent in 2011-2012, exceeding the target by 3 percent. The gender parity index was 1.1 in 2012 in favor of girls, compared to 0.97 in 2005.
    • Concerning retention, the primary completion rate improved significantly from 48.7 percent in 2004 to reach 66.9 percent in 2011-2012, surpassing the 65 percent target. The transition rate between primary and secondary education also surpassed the project targets at project closure (59.6 percent versus a target of 57 percent). This indicates an increase in retention as children can only transition between school levels if retained in the system. By contrast, dropout rates at the primary level, although improving from 13 percent to 9 percent, fell short of the revised target set at restructuring (5 percent).
    • While the extent to which these national level improvements can be attributed to the project is unclear (there is no indication of the proportion of total national schools or students which benefited directly from the project), the ICR makes a reasonably convincing case that, without project-financed improvements, the achievements would have been less. The construction of 28 new secondary schools contributed directly to the improved transition rate. The provision of better learning materials together with improved teaching practices, as well as improvements in infrastructure at both primary and secondary levels, can reasonably be inferred to have encouraged both higher enrollment and retention rates (for example, it is highly likely that the provision of proper sanitary facilities would lead to greater willingness on the part of families to enroll girl students in the schools and keep them there). Summer courses and special attention to students facing difficulties would also be expected to bolster retention rates.

Outcome: Improved Education Quality
Here also the evidence provided is partly at a national level, though it is also drawn from sample surveys of the impact of school projects on the quality of learning:
    • Repetition rates at the primary level were significantly reduced from 13.9 percent at the beginning of the Project to 3.2 percent by project completion (versus a target of 6.3 percent).
    • The improvements in dropout and transition rates mentioned above under intermediate outcomes are also indicators of enhanced education quality.
    • Once again, the ICR makes a reasonable case for partial attribution of these results to the project. It argues that without project interventions at the national level (for example, provision of pedagogical material, training programs, and improvement of the learning assessment system), at the regional level (regional training plans, regional quality plans), and at the local level (the development and implementation of school projects, training of principal, teachers, and School Management Committees), which took place on a fairly substantial scale, it is reasonable to infer that the cited improvements in quality indicators would have been lower.
    • With regard to the surveys, the ICR (page 29) reports that “evaluations of the impact of the school projects on the quality of learning (which were undertaken in 2009 and 2011) on a representative sample of schools showed a positive impact on students’ achievements.” These evaluations showed enhanced achievements in mathematics and French, notably in the early grades. Grade 3 students in schools which received the school project grants performed substantially better in these subjects than Grade 3 students in control schools. In the south of the country. As the ICR puts it, “this observed difference in performance was large amounting to over 30 percent of a standard deviation for mathematics and over 40 percent for French…..The report concluded that a well-targeted program improving resources for schools (as a permanent increase in school spending) would likely have an important effect on student performance.” Little information is provided in the ICR concerning the quality or methodology of the surveys, though they were financed by the project, and appear to have involved the use of control groups.
    • According to the ICR Data Sheet (ICR, page 8), the percentage of children at the elementary level receiving the minimum acceptable scores in reading, languages, mathematics and science had risen from 51 percent for French and 56 percent for mathematics in 2004, to 75 percent in 2009. It is unclear if this result (which is not mentioned again in the ICR) comes from one of the surveys or from elsewhere. In any event, it seems a major change in a short time frame, particularly since the regional training and quality programs were apparently not focused on classroom teachers and did not include specific techniques to improve reading and comprehension.
    • The Summary of the Borrower’s ICR (ICR, page 51) presents the results of project-supported remedial courses which took place over the summer of 2010 and 2011 for students with academic difficulties. In 2010, 11,802 students, and in 2011, 12,144 students in Grade 4 equivalent, received courses in reading and mathematics. Learning assessments at the beginning and end of the program show that in 2010, the success rate went from 40% to 66% and from 51% to 75% respectively in French and mathematics respectively. In 2011, similar positive results were observed.

Progress Towards Meeting Program Goals
It is apparent from the above that progress was made during project implementation towards the realization of the universal education objectives of the Ten Year Education and Training Program (2005-2015). Although the program is not complete, there were no triggers for a third phase of the APL, and no such phase was in operation at the time of preparation of the ICR.

5. Efficiency:


Efficiency is rated modest.
    • A cost effectiveness analysis was carried out at appraisal, in addition to a fiscal impact and sustainability analysis. In the absence of available household data, there was no economic analysis and a Net Present Value and Economic Rate of Return were not calculated.
    • An analysis of the social and private rates of return at closure was carried out using the 2011 Household Survey. The Survey shows a positive impact of primary and secondary education on health outcomes (for example, infant mortality, nutrition, and knowledge of HIV-AIDS). The social return on primary education is estimated at 4.9 percent and the private return at 15.9 percent. The returns at the junior secondary level are 11 percent and 13 percent respectively. However, little detail is provided on the methodology behind these calculations -– for example, what is the null hypothesis? Is it no primary or secondary education? If so, its applicability to the project is limited since the project supported the improvement of an already existing education system rather than its provision from scratch.
    • The ICR states (page 32) that the project has been efficient in delivering most of the expected outputs. Targets set at appraisal, for example the number of schools to receive grants and the number of adults trained in literary classes, were exceeded, although the resources spent did not increase. As measure of efficiency, however, this depends on the reliability of the initial estimates.
    • The ICR provides limited information on unit costs, comparing the FCFA 98 million for a government-built junior secondary school (as under the project) with an FCFA 105 million school under a USAID project. This relatively small difference could be explained by a number of factors that have little to do with efficiency. Also, in a footnote, the ICR states that “the USAID-financed construction program is managed by the same government entity as for the IDA project. The difference is the inclusion by USAID of community mobilization funds in the school construction program.”
    • There were some operational and administrative inefficiencies, such as delays in construction activities and in the acquisition of manuals. There were also procurement-related weaknesses at the local government level. These were reflected in a 20 month extension of the closing date (40% of the original time estimate).
    • No explanation is provided in the ICR for the fact that operational and maintenance costs at closure were nearly double the appraisal estimate.

      a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of objectives is rated high and that of design substantial. Although not all of the recorded improvements in access and retention, and in the quality of education, can be attributed to the project, the evidence of a positive impact arising from project activities is fairly convincing. Efficacy is therefore rated substantial. Efficiency is rated modest. Overall outcome is assessed as moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:


The Government is strongly committed to a long term education plan, is strongly supported by external partners, and has increased the education budget’s share of recurrent expenditures since 2004 from 31.3 percent to 34 percent. Institutional capacity has been strengthened. However overall fiscal constraints and insufficient number of trained personnel still constitute moderate risks for the education program. Areas affecting the quality of education which were still receiving inadequate resources at project closure included weak pedagogic support, lack of information technology, and school grants beyond those financed by the project. A shift in emphasis away from basic education to secondary and higher education may mean that gains made under this project are not sustained.

In addition, the Summary of the Borrower’s ICR (ICR, page 52) points out that “the implementation of the policy of universal education [through expanding the number of state schools] faces resistance among certain groups in areas of the country where the religious schools are dominant.”

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

The project design was relevant with components and sub-components appropriately designed to achieve the project development objectives. It was based on Phase I of the Government and World Bank supported program and focused on a smaller number of activities in areas of the education system which were not being addressed by other external partners.

The project was based on an analysis of education sector needs, thereby ensuring a fit between project and overall APL goals on the one hand and strategic relevance and approach on the other. There was a focus on under-served rural students and those with learning disabilities. Design also emphasized the needs of women and girls.

Project design benefited from lessons learned from Phase One of the Adjustable Program Loan. It included measures aimed to strengthen the administrative and oversight capacities of the Ministry of Education. Some of the activities had been piloted under Phase I and implementation arrangements benefited from streamlined procedures as a result of this experience. The involvement of community and local government stakeholders was emphasized during project preparation -- an improvement over the preparation of Phase I of the program.

The project design utilized numerous relevant government and education agencies for implementation. While this had the advantage of involving a wide range of stakeholders, it also created coordination issues. Annex 6 of the PAD ("Implementation Arrangements") recognized these inherent difficulties in the light of capacity shortcomings in the Ministry of Education and weak coordination between the Ministry and its partners. The risks identified in Annex 6 of the PAD materialized during implementation -- for example local governments were not able to implement upgrading schools. Overall, the implementation framework was well prepared if slightly optimistic.

There were a number of weaknesses in M&E design (see Section 10 below).

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

The World Bank team was able to react pro-actively within its supervision mandate to focus on development impact, working to re-structure the project when necessary, with appropriate performance reporting and supervision of fiduciary and safeguard aspects. For example, it followed up on possible adverse environmental effects resulting from construction (see Section 11 below). The supervision team was adequately funded and had continuity in project management staff, with two task team leaders who worked closely together. The team carried out regular supervision missions and informal consultations between missions, which were well staffed. Given that much supervision was country-based, close collaboration with the Government was maintained. The team provided leadership in guiding external partner support strategies for the overall education program.

The IDA project team downgraded the project ratings to moderately satisfactory in supervision reports between June 2008 and June 2010, when some activities faced implementation difficulties. It also carried out a mid-term review and identified with the Government the need to adjust the indicators and re-structure the project in 2010.

Changes made to reporting requirements after the 2010 restructuring improved project monitoring (see Section 10 below).

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The ICR reports that Government ownership of the education program was strong during preparation and implementation. The enabling environment and policies were also strong. For example, the Government committed more resources than planned to activities under Phase I, financed background studies, and continued to invest in education throughout implementation. It also created a mechanism for ensuring external partner coordination around the goals of its education program. Counterpart funding for Phase II was in excess of the appraisal estimate. The ICR reports that the dialogue with the IDA supervision team and other donors was productive at entry and throughout the project implementation period. Stakeholder consultations were expanded for the preparation of this phase of the APL.

The Government's own ICR and its response to the Bank's draft ICR were frank and constructive in identifying issues and suggesting ways forward for future programming.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:

The implementation arrangements built upon those established for the first phase of the APL. The project was implemented by various Directorates of the Ministry of Education, including the Directorate of Planning and Education Reform, and the Directorate of Administration and Management, along with the sub-sectoral education Directorates. These agencies were coordinated overall by the Directorate of Planning and Education Reform. The Minister of Education had overall responsibility for sectoral and policy coordination and chaired a Steering Committee consisting of the Directors and Heads of involved agencies. Fiduciary responsibilities were carried out by the General Administration and Equipment Directorate of the Ministry with the assistance of contracted regional consultants. The Ministry's regional offices were responsible for implementing quality improvement activities. Implementation was carried out by Ministry staff and there was no project implementation unit.

These implementation arrangements were integral to the goal of mainstreaming and decentralizing education system functions. For the most part, agencies and offices collaborated and functioned well. However capacity varied between agencies and between the national and local levels, leading to implementation difficulties and delays. There were delays in the construction activities and in the acquisition of manuals and some weakness in terms of procurement at the local government level. However, this improved after responsibility for this latter had been transferred to Regional Local Development Agencies which had prior experience of World Bank projects.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The PAD includes an agreed M&E and results framework. This was connected to a comprehensive education sector matrix which was developed by the then named Education for All Fast Track Initiative (EFA-FTI - now called the Global Partnership for Education). Each component also had specific intermediate outcome indicators. However, the results framework was overly complex and inappropriate for project activity monitoring.

There were originally 16 project development objective indicators and 8 intermediate outcome indicators. Several of these were changed or dropped and others were introduced at the first re-structuring in 2010. Many of the original project development indicators were program wide, but not specific to this phase of the program. Others were outdated or could not be measured due to changes over the course of the project.

Some of the intermediate outcome indicators are actually output indicators (for example the number of learners enrolled in literacy classes, the percentage of block grants financed, the successful conclusion of the annual statistical campaign). Together with the fact that some indicators were not monitored or dropped, this meant that for a number of project activities there was no relevant outcome indicator, in particular the teacher training activities and measures for assessing improvements in accountability at the local level.

The Directorate of Planning and Education Reform (DPRE) was responsible for leading and coordinating M&E activities with inputs from all levels of education authority -- the central Ministry, and regional and local levels of government. Data collection was to be monitored by school and district level officials.

b. M&E Implementation:

Key monitoring reports were supported by the project:
1. Annual Economic and Financial Report, which reviews the performance of the education system.
2. The Annual Education Status Report on the performance of different levels of education provision.
3. Other evaluations on the implementation of school projects and learning assessments.

Standardized learning outcomes tests were developed and run twice. Results were shared with teachers and inspectors and used to guide new interventions, although it is not clear what these were. Some regions started working together to harmonize the content and timing of these tests and to institutionalize them.

Overall, there seems to have been few major issues with the implementation of these activities, with the exception of the learning outcomes test which was delayed and only run twice over the course of the project. The Government carried out regular monitoring activities to gather feedback on progress and adjust the program. It commissioned independent reviewers to evaluate the Ministry of Education.

However the number of monitoring indicators that were not utilized was significant. The project contained too many and too high-level indicators for implementing staff to track, particularly given the decentralization of some activities and the training and time needed to do this adequately.

Changes made to the reporting requirements after re-structuring enabled the project monitoring to improve. However, this was too late to measure some major activities.

a. M&E Utilization:

Data have been used for assessing overall progress on the long term government program and the ICR notes that improvements in national data collection are significant. These have resulted in the adjustment of program activities, a specific case being the use of data to refocus school grants after the first phase.

The project contributed to creating a climate in which the monitoring and evaluation of learning achievements is becoming the norm, keeping the focus on quality. However, although standardized test results have been used as diagnostic tools to assess priority areas for project activities, data usage from evaluations had some weaknesses. There was insufficient sharing of results with schools and no systematic effort to disseminate the results.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The project was given a Category "B" rating for purposes of environmental assessment. In addition to Environment Assessment (OP 4.01), Involuntary Resettlement (OP4.12) was triggered "due to potential negative environmental and social impacts related to the construction of schools" (PAD, page 16). According to the PAD, an Environmental and Social Management Framework (ESMF) was prepared during appraisal to mitigate any negative effects from school construction because the precise location of the schools and any local impacts were not identified prior to appraisal. A Resettlement Policy Framework (RPF) was also prepared. Both the ESMF and the RPF were publicly disclosed before appraisal.The Ministry of the Environment was charged with supervising implementation of the ESMF. It is not clear from the ICR if specific environmental clauses needed to be addressed during bidding. The Project Implementation Manual contained a chapter on environmental and social management.

The ICR (page 23) reports that no adverse environmental effects occurred as a result of the construction of secondary Schools, and that the School Construction Department followed up on aspects of construction that needed attention, specifically site clean-up and landscaping. A collaboration agreement between the Directorate of Planning and Reform of Education and the regional Directorate of the Environment was signed before project closure. The ICR contains no clear statement that OP 4.01 was complied with.

Although OP 4.12 was triggered at appraisal due to the potential need for land acquisition which might have involved resettlement, the ICR reports that, in the event, school construction did not involve any resettlement.

b. Fiduciary Compliance:

Financial Management: A Financial Management Assessment was carried out as part of project preparation and minor failings were addressed. The ICR reports that Financial Management of the project was rated satisfactory in supervision reports and all audits were unqualified. Auditors' recommendations were carried out before project closure (ICR page 22).

Procurement: Procurement at the central level was satisfactory with an experienced team in charge of project procurement. However difficulties were faced in the procurement of textbooks. There were issues at the local government level in carrying out construction activities. This responsibility was transferred to the regional development agencies as there was insufficient capacity at the municipal level. There were no reported cases of misprocurement.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Negligible to Low
Moderate
Although there is strong Government commitment to improve education, further investment is needed in the education sector in order to meet the Millennium Development Goals. Moreover project results could be prejudiced by competing investment priorities, a shift towards secondary and higher education and/or slow economic progress and the persistent lack of qualified teachers. 
Bank Performance:
Satisfactory
Moderately Satisfactory
M&E design had weaknesses that caused difficulties throughout the first phase of project implementation and at evaluation. The assessment of local implementation capacity was over ambitious leading to delayed implementation. 
Borrower Performance:
Satisfactory
Moderately Satisfactory
There were capacity weaknesses in some levels of the education system. There were delays in the construction activities and in the acquisition of manuals and some weakness in terms of procurement at the local government level.  
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
    The following lessons can be drawn from the experience of preparing and implementing this project:
      • Educational quality is likely to be more effectively enhanced not only through testing for learning outcomes, but also through providing for interventions that address problems identified during the testing process.
      • Unless an education system is supported through all its levels, systemic change is not likely to be achieved. In this case, although training for central government offices was appropriately provided for, activities to support regional and local education offices were insufficient.
      • Regular monitoring and evaluation contributes to the achievement of sought outcomes. Although there were some data weaknesses and insufficient dissemination of results, the achievements of the Ten Year Program were independently evaluated at the end of each phase and lessons extracted for future application.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR adequately focused on outcomes, although its use of evidence to support conclusions on project impacts could have been strengthened. It is candid about the flaws in project implementation. It covers the background context and justification for the project well. The ICR presents a clear and helpful picture of how its ratings were assessed in Annex 10. There were some internal inconsistencies, notably between the number of components in the main text and in Annex 1. There was no clear statement of compliance with safeguards policies.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions