Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Sri Lanka Education Sector Development Project


  
1. Project Data:   
ICR Review Date Posted:
09/26/2012   
Country:
Sri Lanka
PROJ ID:
P084580
Appraisal
Actual
Project Name:
Sri Lanka Education Sector Development Project
Project Costs(US $M)
 60.0  72.41
L/C Number:
C4423, CH196
Loan/Credit (US $M)
 60.0  72.41
Sector Board:
Education
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  02/23/2006
 
 
Closing Date
06/30/2011 06/30/2011
Sector(s):
Primary education (45%), Secondary education (35%), Tertiary education (8%), Adult literacy/non-formal education (6%), General public administration sector (6%)
Theme(s):
Education for all (100% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Maurice X. Boissiere
Judyth L. Twigg Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The Project Development Objectives (PDO) as stated in the Credit Agreement and Project Appraisal Document (PAD) were to: (a) promote equitable access to basic education (grade 1-9) and secondary education (grade 10-13); (b) improve the quality of education; (c) enhance the economic efficiency and equity of resource allocation and distribution within the education system; and (d) strengthen education governance and service delivery.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project followed a Sector Wide Approach (SWAp) with four themes stated in the PDO. IDA funding was integrated with Government funding, so the allocations for the four themes are only indicative. The difference between costs at appraisal and actual costs was due to fluctuation in the dollar-to-SDR exchange rate.

The project had four components derived from the themes stated in the PDO.

Theme 1 Equitable access to basic and secondary education (US$ 15 million appraisal, US$ 18.2 million actual): . Four key initiatives were provided for all children for basic (grades 1-9) and secondary (grades 10-12) education: (1) demand side incentives for school attendance, (2) upgrading the school network to relax constraints on geographically equitable access to good quality basic and secondary education, (3) strengthening special education programs for children with special learning needs, and (4) strengthening non-formal education programs for adolescents not in school

Theme 2 Improving quality of education (US$ 15 million appraisal, US$ 18.2 million actual): This component aimed to improve learning outcomes by developing generic skills, such as creativity, communication, problem solving, and non-cognitive achievements. There were three policy initiatives: (1) restructuring and upgrading curriculum, (2) teacher development, and (3) modernizing exams and testing.

Theme 3 Improving economic efficiency and equity of resource allocation (US$ 15 million appraisal, US$ 18.2 million actual): There were three policy initiatives under this theme: (1) establishing a medium term budget framework. (2) developing an overarching education sector development plan, and (3) establishing a public expenditure and quality education tracking system.

Theme 4 Improving governance and delivery of education (US$ 15 million appraisal, US$ 18.2 million actual): There were three policy initiatives under this theme: (1) establishing a balanced control model of school based management, (2) conducting organizational analysis for capacity building, and (3) implementing a human resource development strategy.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs: The Bank credit was US$ 60 million equivalent at appraisal. Total project costs were US$72.41 million equivalent, entirely financed by the Bank credit. The credit was for US$70.0 million, but fluctuation in the dollar-to-SDR exchange rate brought the the cost to US$72.41 million.

Financing: The Bank credit was US$ 60 million equivalent at appraisal. In 2008 the Bank credit was increased by US$ 10 million for a total of US $70 million. This was to allow for more resources to the areas in the North and East that were affected by the ethnic conflict. The ICR stated that other donors such as the Asian Development Bank, United Nations Children's Fund, and many others supported the SWAp and would offer parallel financing, but the amount of this financing was not stated.
Borrower Contribution: Since the project used a Sector Wide Approach (SWAp) that encompassed the whole of basic and secondary education, according to the Education Sector Development Policy Framework, the Borrower contribution was essentially the Government budget for education at the basic and secondary levels. The PAD estimated that the education budget over the lifetime of the project would be US$ 2,720 million. The ICR did not present an actual amount of the Government budget over the project lifetime.

Dates: The project closed on schedule on June 30, 2011, with full disbursement of the Bank credit (original and increase).


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
The relevance of project objectives is rated substantial. The objectives were relevant to the country context at appraisal when the challenge was to improve education access and quality while the country was undergoing the separatist conflict in the North and East. They were relevant to the most recent Country Assistance Strategy (CAS, FY 2009 - 12) at the time of completion. This CAS had three central themes: Peace, Growth, and Equity. These themes supported the country’s Poverty Reduction Strategy. Concerning the education sector, the CAS addressed two focus areas: (1) reducing regional inequities in the provision of quality basic education, and (2) improving curricula in secondary education. The objectives were also relevant to the country’s own strategic vision (“Mahinda Chintana: Vision for the Future”) that included all of the education themes stressed in the objectives.

b. Relevance of Design:
The relevance of project design is rated substantial. A Sector Wide Approach (SWAp) was appropriate for implementing wide ranging policy objectives. A regular investment project with ring fenced activities separated from the normal operations of the sector was considered. However, this approach would not have been able to achieve as much in the policy arena. A Development Policy Loan was also considered. However, this approach would have been too short term for some of the long-term changes needed such as curriculum reform, teacher development, etc. The results chain linking desired outcomes of access and quality to inputs such as demand side incentives to families, teacher development, curriculum improvement, and school-based management was sound and based upon the latest results of education research and practice.


4. Achievement of Objectives (Efficacy) :

Each of the four development objectives were substantially achieved. Given the improvements in education outcomes, and the timing of the interventions (inputs and outputs) and change in results from the start to the end of the project, attribution of these improved results to the project's interventions is likely, For example, a rigorous impact evaluation of the Program for School Improvement (PSI) showed that PSI schools improved student learning more than those without PSI. Comparing project indicators, as well as extensive sector reports at the start and the end of the project period, also gives evidence for positive contribution of the project to achievement of the objectives. However, previous projects and policy interventions may also have made some contributions.

(1) Promote equitable access to basic education (grades 1-9) and secondary education (grades 10-13) (substantial)

Outputs
Demand side measures were introduced (such as school feeding) to encourage attendance, especially among the poorest households.

Supply side measures were introduced to upgrade schools (such as sanitation measures discussed below) and to make attendance easier and more attractive. Primary schools are available within 2 kilometers of every household (source in addition to ICR : “Transforming School Education in Sri Lanka," Sector Report, World Bank, 2011).

A National Survey of out-of-school children was conducted by the Ministry of Education.

School attendance committees were established within each district to promote school attendance, especially for special needs and non-formal education, among children ages 6-14.

Secondary schools in the rural areas benefitted from upgrades in information technology, science lab equipment, and sanitation that were part of the demand-side effort to increase access by making schools more attractive.

Expansion of the network of secondary schools was undertaken such that, by 2010, there was a secondary school within five kilometers of every child's home (source: “Transforming School Education in Sri Lanka," Sector Report, World Bank. 2011).

Outcomes
Net enrollment rates for ages 6-14 could not be calculated at project end due to a delay in the national census caused by the military conflict in the North and East. The net enrollment rate (ages 6-14) was 81% in 2005 with a target of 83% in 2011. However, data from outside of the ICR (“Transforming School Education in Sri Lanka Project," Project Appraisal Document, 2011) showed the net enrollment rate for primary education (ages 6- 10) was 99% in 2010, up from 96% in 2005. Unless there was a decline in net enrollment rates for ages 11-14 (not likely), the net enrollment for ages 6-14 was likely to have risen significantly.

Survival rates from grade 1 to grade 9 increased from 78% at baseline to 91% at project closing, exceeding the target of 88%.

Equity improved with respect to survival rates, with the poorest provinces showing the largest gains. Survival rates improved by 32% in Eastern Province, 26% in Northern Province, and 25% in Uva province, versus a 9% improvement in the wealthier Western province, which includes the capital Colombo. All nine provinces now have survival rates above 80%, whereas six of the nine provinces had survival rate below 80% (ranging from 66% to 79%) in the baseline year 2005.

The number of out-of-school children (age 6 -14) decreased by 67,708 over the project's lifetime, exceeding the target of 50,000. The ICR did not give the beginning and end data points for the out-of-school population.

(2) Improve the quality of education (substantial)

Outputs
New curriculum for grades 1 – 12 was developed and implemented. Emphasis was on improving higher order thinking skills, ICT skills and English language competency. Teachers and in-service advisors were trained in implementing the new curriculum.

The percentage of schools with teaching plans increased from none (2005) to 100% (2011), exceeding the target of 50%. This was done by establishing provincial text book scores, improving text book order registers, streamlining distribution processes, conducting zonal awareness programs of new procedures, and improving coordination/distribution services.

Teacher Instructional Manuals along with Syllabi were revised and distributed to all schools.

Training of In-Service Advisors was dramatically increased from 3,265 in 2005 to 21,779 in 2010, and they in turn helped teachers and principals by providing practical pedagogical and management guidance.

Exam item banks and guidelines for all core subjects (Sinhalese, Tamil, English, Science, Mathematics, History, Five Religions) were developed and sent to all schools by 2011, meeting the target.

Access to bilingual (English) schools increased from 428 schools (in 2008) to 867 (in 2011), falling short of a target of 1000 schools. A shortage of qualified bilingual teachers was a constraint.

Schools with improved health environment increased from 606 (2009) to 4,029 (2011), exceeding a target of 750. Schools provided with water and sanitation increased from 695 to (2009) to 1,984 (2011), exceeding a target of 750.

78 of the 100 Teacher Centers (exceeding the target of 60) conducted multi-ethnic teacher training to promote social cohesion through the school curriculum.

The National Institute of Education established committees for promoting multi-ethnic activities in schools and produced two review reports on civics and history curricula for secondary education.

The percentage of schools with certified (or graduate) teachers and high quality teacher development systems increased from 80% (2005) to 95% (2011), exceeding the target of 90%.

The percentage of textbooks delivered to schools on time increased from none (2005) to 100% (2011), exceeding the target of 60%.

Exam item banks and guidelines were specified and sent to all schools by 2011, meeting the target.

Outcomes
In grade first language, the percent of pupils scoring above 50% on the National Assessment of Learning test increased from a baseline of 69% (in 2005) to 83% (in 2009), exceeding the target of 73%.

In grade 4 mathematics. the percent of pupils scoring above 50% on the National Assessment of Learning test increased from a baseline of 67% (in 2005) to 82% (in 2009), exceeding the target of 71%.

In grade 4 English, the percentage of pupils scoring above 50% in the National Assessment of Learning test increased from 32% to 58%, exceeding the target of 35%.

In grade 4 for schools in Program for School Improvement, mathematics and English scores increased by 0.20 and 0.18 standard deviations respectively, compared to other schools. No target was specified.

In grade 8 mathematics, the percent of students achieving scores higher than 50% increased from 33% (in 2005) to 45% (in 2008). In grade 8 science, the percent achieving scores higher than 50% went from 53% (in 2005) to 59% (in 2008).

In secondary grades 10-13, there are no data from the National Assessment of Learning (grade 10 will be assessed in 2012). However, the national examinations conducted at O-level (after grade 11) show pass rates of 48% (in 2005) and 49% (in 2009). Pass rates for A-level (after grade 13) were 60% in both 2005 and 2009.

(3) Enhance the efficiency and equity of resource allocation and distribution within the education system (substantial)

Outputs
A Public Expenditure and Quality Education Tracking System was designed and established that allowed for the first time tracking of expenditures in terms of cognitive learning goals and equity goals by provinces; a Medium Term Budget Framework was developed and established; and an overarching sector development plan was prepared and implemented.

Outcome
The share of recurrent budget for higher order learning increased from 0.5% to 3.3%, exceeding the target of 3%. The share of capital budget for higher order learning increased from a baseline of 40% to 43%, exceeding the target of 42%. The budget share for maintenance/repair of capital stock increased from a baseline of 5% to 18%, exceeding the target of 15%.

Resources were increased for provincial schools through re-allocation of funds from the national schools budget, but the ICR does not state by what amount or percentage.

The increase in resources to the poorest provinces is likely reflected in the improved performance of the poorest provinces in survival rates (see above) and also in the National Assessment of Learning for grade 4. For example, data in the ICR Annex 2 show the wealthiest Western Province increased its mathematics score only by 6%, whereas Northern Province increased by 49%, Eastern Province by 35% and Uva Province by 29%. In achievement in first language (Sinhalese or Tamil) the wealthiest Western Province increased its score only by 5%, whereas Northern Province increased by 38%, Eastern Province by 26%, and Uva by 25%.

(4) Strengthen education governance and service delivery (substantial)

Outputs
The Program for School Improvement (PSI) to introduce school-based management was tested and established in all schools, meeting the target.

Organization capacity reviews of MOE/Provincial Offices were conducted for all provinces, meeting the target.

A long-term human resource plan for MOE/Provincial Offices was completed/implemented, 80% for MOE and 60% for provinces.

Training and workshops were provided for 110,501 individuals in various responsibilities including school leaders, provincial and national planning and budgeting officers, and in-service advisors. There was also long term training abroad for 18 professional staff, resulting in master’s degree or post-graduate certificates in education planning, management, curriculum and instruction. Short term training and study visits abroad were provided (in comparable Asian countries) for 81 MOE staff in leadership development, financial management, competency based curriculum, and other project relevant topics.

Outcomes
A strong planning culture was established, with collaborative processes between the national and provincial levels; capacity now exists to analyze expenditures in both recurrent and capital items in relation to education goals and results.


5. Efficiency:

Efficiency is rated substantial. Implementation proceeded efficiently with no delays, and the project closed on time. Given that project resources were pooled with the Government budget, the ICR does not isolate the efficiency of project resources. As a result of pooling Government and Bank funds, transaction costs were reduced. The internal rate of return was estimated to be about 16%, which the ICR states is comparable to that at appraisal. Annex 3 (p.41) explains the methodology. The project costs are reasonably well known. The benefits are estimated from labor force survey data concerning wages by levels of education, a fairly standard approach. However, the assumption must be made that the survey data apply, at least to a good approximation, to the beneficiaries of the project.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
%
%
ICR estimate:
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of objectives and relevance of design are substantial. Each of the objectives were substantially achieved. Efficiency was substantial. The project's objectives were responsive to country conditions at the time of appraisal, and are consonant with current Bank strategy and Government strategy. The results chain was sound and based upon the latest education research and best practice. Access to basic and secondary education and improved, and improved test scores were indicative of improved quality of education as well. Governance and service delivery in the sector was enhanced, and efficiency of resource allocation improved.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The ICR presents plausible evidence that It is highly likely the project will sustain outcomes. It states that the Government continues to use the ESDP budgeting procedures as evidence of sustaining the sector approach. This is also an indicator of fiscal sustainability, reducing risk to outcomes. The Government has also prepared ESDP2 with the Bank, which will continue the four themes in the same SWAp modality. However, the ICR identified potential, albeit low risks. Policy support for decentralization must continue. Capacity of the weaker provinces needs continued emphasis. The division of roles and responsibilities of the center versus the province could be complex and provides some risk.

a. Risk to Development Outcome Rating: Negligible to Low

8. Assessment of Bank Performance:

a. Quality at entry:
Extensive sector review was undertaken prior to project approval. The executive summary of the sector review was widely disseminated in Sri Lanka (Sinahala, Tamil, and English), promoting public awareness and ownership . Special studies were undertaken on the examination system, human resource development, teacher development, procurement, and monitoring and evaluation. The Quality Enhancement Panel assessed the preparatory work and draft PAD as excellent, including design of M&E (see M&E section). Other donors supported the approach and expressed intentions to provide parallel financing for the education sector-wide approach. The project identified risks, and adequate mitigation measures were taken, For example, initial misunderstanding of the Program for School Improvement was resolved through public dialogue and obtaining Government approval at the Cabinet level.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:
The Bank team provided intensive and frequent supervision that focused on outcomes. A set of surveys and studies was included in the project to obtain relevant and reliable data. Local professionals were engaged to assist the Bank team to provide widespread and frequent supervision, monitoring and help in solving implementation issues. One issue was in the fiduciary area that involved audit opinions that were qualified for a period due to weak asset management. However, the ICR pointed out that this was a minor issue that was corrected by end of the project by adding more government staff. The ICR also stated that more Bank support could have been provided in this area, and more will be done in the follow-on project.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The ICR distinguishes between Central and Provincial Government performance. The Central Government showed strong commitment to the project by supporting decentralization of its administrative powers to the province. Through the Finance Commission, which comes under the Ministry of Finance and allocates budgets to the provinces, flow of funds proceeded efficiently to the provincial level. The Government also exercised leadership through a series of public consultative meetings to raise awareness of the new education sector policy, especially decentralization and the Program for School Improvement. All of this contributed to ownership at the provincial levels. Apart from the minor issue of delay in staffing to keep adequate asset management records, there were no major fiduciary issues.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:
The MOE and the Finance Commission effectively coordinated and monitored the overall project and assisted implementation by the Provincial Education Authorities. A high level committee chaired by the Secretary of MOE held monthly meetings to monitor activities and resolve issues. Visits to the provinces were conducted every two months by the MOE teams (together with the Bank team). At the provincial level, the Provincial Education Authorities followed through on their responsibilities, especially in the Program for School Improvement and the capacity development activities. The focus was on the project objectives and outcomes and necessary actions to achieve them.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The Quality Enhancement Review panel considered the M&E design “very thorough and comprehensive." The Results Framework of the PAD and the numerous indicators (4 outcome and 15 intermediate outcome indicators listed above) were in good correspondence. Baseline information was available for the indicators, and it was clear who was responsible for M&E data. The schedule of data collection was adequate to help monitor progress.

b. M&E Implementation:
The data needed were collected periodically by different groups and agencies, in some cases by academics, in other cases by consultants. The two main agencies were the Ministry of Education for the national level and the Finance Commission for the provincial level. In some cases, special surveys and studies were done, such as the island-wide survey for out-of-school children. The ICR lists 34 special studies supported by the Project (see Annex 2, Table 2.28). Among these studies, the National Education Research and Evaluation Center conducted the periodic learning assessments. There was also an expenditure tracking system that was developed and used by the Finance Commission within the Treasury Department that allocates funding to the provinces.

a. M&E Utilization:
The data collected were used during implementation by providing the basis for evidence-based policy dialogues among the MOE, Finance Commission, Provincial Councils, and the Bank team. The Government and Bank team (resident mission) met monthly to monitor progress, using data such as Expenditure Tracking, and official missions (twice annually) boosted implementation of any mitigation measures, Near close of the project, the M&E system provided useful data for preparation of the follow-up operation.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:
One environmental safeguard issue (OP4.01) arose due to plans for construction of schools and a warehouse for textbooks. All of this construction followed Bank operational policies, and there were no major environmental issues. There was compliance with environmental safeguards.

b. Fiduciary Compliance:
Procurement and financial management were rated satisfactory throughout the project, with an exception that weak asset management resulted in qualified audits for a time. This was the result of a shortage of Government project staff, and was rectified by the end of the project.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Moderate
Negligible to Low
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The ICR (page 18) drew the following lessons.

1. Using the country system for a SWAp can increase ownership, but a strong M&E system is also necessary. In this project, a strong M&E system provided data to high level officials who demonstrated ownership in their monthly monitoring and decision meetings.

2. Decentralization enhances ownership and efficiency, but building local capacity of local officials is essential for success. In this project, the Technical Support Unit from the national Finance Commission provided the support needed to build capacity at the provincial levels.

3. Involving parents, graduates, and local communities in school management can lead to more accountability and efficiency of service delivery. In this project, the impact evaluation of the Program for School Improvement showed that quality and efficiency of education services were improved as a result of these measures.

4. Periodically monitoring and evaluating the progress of the sector can strengthen the education policy framework. In this case, the project had the advantage of being sandwiched between two major sector reviews. This helped in the evaluation of this project as well as the preparation for the follow-up project.

14. Assessment Recommended?

Yes
Why?
A PPAR has been launched because the Sri Lankan education sector has not yet been evaluated by IEG, and this operation is the first time a SWAp has been done in Sri Lanka.

15. Comments on Quality of ICR:

The ICR is clearly written, following the major points of the guidelines. Evidence is presented for all of the numerous indicators. However, net enrollment rate was lacking for the end of the project (2011) because the census was postponed due to the civil war. The ICR does not explain how the base year (2005) net enrollment rate was estimated and does not explore if other methods could have been tried for the end of project estimation. The Region subsequently clarified that the net enrollment rate in 2005 was estimated based on a 2005-2006 Household Income and Expenditure Survey (HIES), and that the ICR was completed just before 2011 HIES and census data were released. The ICR emphasizes results and uses the indicators, as well as other information collected, appropriately to analyze the ratings for the project.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions