|1. Project Data:
ICR Review Date Posted:
|Third Basic Education Quality Improvement Project
Project Costs(US $M)
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|Primary education (54%), Pre-primary education (20%), Tertiary education (12%), Central government administration (11%), Other social services (3%)|
|Urban services and housing for the poor (23% - P)
Rural services and infrastructure (22% - P)
Education for all (22% - P)
Vulnerability assessment and monitoring (22% - P)
Participation and civic engagement (11% - S)|
||ICR Review Coordinator:
||Judyth L. Twigg
||Lourdes N. Pagaran
|2. Project Objectives and Components:|
According to the Legal Agreement of June 17, 2002, the objective of the Third Basic Education Quality Improvement Project was "to increase the equity, quality and efficiency in the provision of preschool and primary (i.e., ages 4 through 11) education in Uruguay." This was to be achieved through: "(i) the expansion of the full-time school model, focusing on students from socio -economically disadvantaged backgrounds; (ii) the enhancement of the teacher training system and the introduction in the classroom of new teaching and learning instruments; and (iii) an increase in the efficiency of primary education institutions" (Schedule 2, page 12).
The wording of the Project Development Objective on page 3 of the Project Appraisal Document (PAD) was shortened but is consistent in substance with the legal agreement: "to increase equity, quality and efficiency in the provision of preschool and primary education. These objectives will be achieved by: (i) expanding the full-time school model, which focuses on students from socio-economically disadvantaged backgrounds; (ii) improving the quality of preschool and primary education by enhancing the teacher training system and introducing new teaching and learning instruments in the classroom; and (iii) increasing the efficiency of education institutions."
The project was re-structured four times (see Section 2d). The Project Development Objective (PDO) remained the same. However, in 2006 the project's focus on expanding Full Time Schools (FTS) to rural areas was broadened to include students in non-FTS time schools in urban areas. Changes included reducing the outcome targets from 86,000 student places in Full Time Schools to 47,000 places, and the introduction of an alternative model of school to address those disadvantaged children in non-FTS.
b. Were the project objectives/key associated outcome targets revised during implementation?
This project was the third in the series "Basic Education Quality Improvement Projects," through which the World Bank supported the education strategy of the Government of Uruguay. Despite high primary completion rates in Uruguay, there were still issues in the quality of education for disadvantaged students. In order to address this issue, the Government had put in place a system of Full Time Schools (FTS), designed to channel resources to disadvantaged students. The Task Team Leader explained that the FTS model includes: extending the number of hours in the school day from about 4.0 to 7.5; introducing a new teacher development program; improving educational infrastructure; providing meals and nutritional snacks; and introducing children to a broader curriculum. Other features of the model relate to pedagogical approaches, including the promotion of an overall school environment with clear agreed norms within a supportive atmosphere. The model not only addresses learning achievement but reducing repetition and violence in urban contexts. At the preschool level, coverage among 5 year olds is 95.5%, while coverage among 4 year old children is 94.5% across all schools in Uruguay. In FTS preschool children have an extended day, improved curriculum, and other interventions.
A 1999 Student Assessment for 6th graders indicated that disadvantaged students in the FTS program at that time did better than those in regular schools, indicating that the FTS system had a positive impact on learning outcomes for those students. The Government thus decided to expand and improve the FTS system in order to further target disadvantaged students, and the approach is now part of the overall government education strategy rather than a project-specific initiative.
Within this context, the project had two components:
Component 1. Expansion of the Full-Time School Model (US$44.52 million at appraisal, US$120.47 million at closure). The aim was to expand an existing system of Full-Time Schools (FTS) and introduce innovations to make the system more effective. This component had four sub-components:
- construction, transformation, and rehabilitation of full-time schools and the institution of school maintenance funds to prevent the deterioration of public assets;
- acquisition of school equipment, learning materials, and school libraries;
- teacher training on the full-time school model, including support for disadvantaged students, new educational technologies, and bilingual education. and
- strengthening partnerships between schools and parents.
Component 2: Institutional Strengthening (US$11.06 million at appraisal, US$16.05 million at closure). This component aimed to strengthen both individual schools and the education system as a whole. It had four sub-components:
- reforming the system of teacher training;
- the development and implementation of Education Improvement Projects (PMEs);
- monitoring and data collection in the education program; and
- project administration.
Post-restructuring, the components remained the same, but the focus of activities changed, primarily broadening the type and number of schools involved in the project and changing from reaching the disadvantaged in rural schools to impacting most schools with disadvantaged children where the most extreme poverty and violence exist, rural or non-rural (known as Priority Attention Schools). These are referred to in this review collectively as non FTS, the term used in the ICR. A number of activities were scaled up and/or mainstreamed in 2006 and therefore were no longer financed under the project. In its final years, the project concentrated on in-service teacher training and infrastructure activities.
2006 Changes to Project Activities
- the provision of technological equipment, textbooks, and learning materials was expanded to include regular schools that serve disadvantaged children;
- teacher training activities were expanded to teachers and staff in all schools with disadvantaged students;
- bilingual teacher training was discontinued under the project, but scaled up and institutionalized under the Pre-School and Primacy Education Council (CEIP) umbrella;
- training of teachers on new technologies was discontinued under the project and mainstreamed into the regular education program; and
- management information system strengthening activities were completed and therefore did not continue after 2006, with the exception of the international learning assessments.
- construction of a teacher training college; and
- scale-up of the number of education improvement subprojects, with emphasis broadened from nutrition to pedagogical and other inputs.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs: Construction costs rose considerably over the course of the project. This increase was due to a change in the Government strategy from classroom rehabilitation to new construction, and significant increases in construction costs in the country. Construction costs rose on average by about 144% during the period 2002-2009. The Uruguayan National Statistical Institute Construction Cost Index reported an increase in construction costs from August 2006 to June 2009 of about 27% (Project Paper on Additional Financing, para.13, page 3 ).
- January 21, 2004: Schedule 1 was amended to increase the disbursement percentage for imported goods and reallocate amounts under existing categories of expenditures.
- February 24, 2010: Additional Financing in the amount of US$29.9 million was approved to address a financing gap that had arisen due to increased project costs.
- June 6, 2012: Funds were reallocated to finance an expansion of the in-service teacher training program and for the hire of additional advisors for the supervision of works.
Borrower Contribution: The Borrower contribution was estimated to be US$ 14.0 million at appraisal. As per the ICR, the contribution was US$65.12 million, or 465% of the appraisal estimate, as the Government mainstreamed and assumed responsibility for several key project interventions.
- July 13, 2006: The project was re-structured to include an extension of the closing date from December 31, 2007 to December 31, 2009, to allow for expansion of activities.
- February 24, 2010: Additional Financing was approved with a closing date of December 31, 2012.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:
Relevance of Objectives is rated Substantial. At appraisal in 2002, the project objective was relevant for the Country Partnership Strategy of 2000, and it remains highly relevant to the current World Bank Country Partnership Strategy (2010), which supports the "Government's efforts to increase equity, quality, and efficiency in the provision of preschool and primary education." At appraisal the objectives were also highly relevant to the country's economic context, where education was almost universally available but where there were specific issues in addressing the quality of education for disadvantaged groups. However the measurability of the first and second objectives, as articulated, was challenging given the lack of clarity in measuring results ( see section on M&E).
The project is also highly relevant to the current priorities of the Government of Uruguay and its education sector reform program, which prioritizes preschool and primary education. The program has received the support of the President of Uruguay. The objective of expanding the FTS approach is currently being accelerated, with 50% of funding coming from the follow-on World Bank-funded operation "Support to Uruguayan Public Schools Project," approved in September 2012, and approximately 80 new schools anticipated over the next four years.
b. Relevance of Design:
Relevance of Design is rated Substantial. Project design was substantially relevant to its objectives, as the components were logically and plausibly linked to the three project development objectives. Inputs to Component 1 were an expansion of activities supporting the Full Time School model. Component 2 contained activities that supported institutional strengthening, chosen to help improve efficiency within the education system (improved teacher training, data collection, and increased local level and community engagement through school projects). Thus the logical flow of the results framework was commensurate with the objectives to be achieved.
|4. Achievement of Objectives (Efficacy) :|
Increase equity in the provision of preschool and primary education by expanding the full time school model: Substantial
The original focus on the Full-Time School model was broadened over the life of the project so that inputs also impacted Quintile I and II students in non-FTS. Additional information was provided by the Region on students in non-FTS. The Region also noted that inputs to FTS and non-FTS uniformly affect pre-school programs.
- 868 new FTS classrooms were built (329 in new schools, 286 new classrooms in existing schools, and the remainder rehabilitated) in urban areas. This is less than the original target of 1,356, but slightly over the 850 target of December 2012.
- 14 FTS rural schools were rehabilitated (discontinued as a target at re-structuring in 2006, after policy changes). According to the Borrowers' comments, 45 FTS were added between 2010 and 2012, and the number of FTS rose from 92 in 2002 to 175 in 2012.
- The Region provided additional information that 207 schools received improvements to pre-school facilities as well other quality-related inputs, providing 16,400 disadvantaged and most disadvantaged students with access to an improved preschool education.
- The number of students in FTS rose from 21,419 in 2002 to 40,156 in 2012. This indicator was partially met, with the original target of 86,000 students revised to 47,000 students in 2012. The achievement below target was the result of the Borrower's decision to limit class sizes to 25 students. In addition, the shift in focus from FTS only to non-FTS schools with a high majority of low income students meant that the indicator does not accurately reflect the overall impact of project activities.
- The number of students from the first and second income quintiles as a percent of all FTS students rose from 47% in 2002 to 52.7% in 2012, only a small increase. The FTS share of all students in the first and second income quintiles more than doubled between 2002 and 2011, from 7.4% to 15.4%.
- The proportion of FTS students as a percent of total urban enrollment in preschool and primary almost doubled, from 6.2% to 11.7%, between 2002 and 2011.
- Access to preschool for children in Quntiles I and II increased from 70% aged 4 and 90% aged 5 in 2002 to 87% and 92.7% in 2012. 62% of these children are enrolled in the FTS and PAS schools. Results from the OECD 2009 PISA tests demonstrate that in Uruguay students that attend pre-school score on average 50 points higher than those who have not.
- The percentage of sixth grade students from disadvantaged and very disadvantaged backgrounds who performed at a satisfactory level changed between 1999 and 2009 as follows: in FTS only, increased from 53.8% to 64.8% for Language, but decreased from 35.6% to 33.4% for Math. For all disadvantaged/very disadvantaged students (FTS plus non-FTS), the performance was better: an increase from 48.8% to 63.1% for Language, and an increase from 27.9% to 31.8% in Math. There was a change in testing methodology in 2005, but presumably that change would have similarly impacted FTS and non-FTS students. It would appear from these data that the FTS system had little discernible advantage over the non-FTS system for disadvantaged students. However, without knowing more about student characteristics in these two systems, it is not possible to say more at this time.
- The equity gap -- defined as the difference in the percentage of students from advantaged socioeconomic backgrounds performing in a satisfactory manner, and the percentage of students from disadvantaged backgrounds performing in a satisfactory manner -- changed between 1999 and 2009 as follows: for disadvantaged students, decreased from 37.9 to 23.1 for Language and 40.2 to 38.3 for Math; and for very disadvantaged students, decreased from 39.2 to 31.9 for Language and increased from 42.0 to 46.4 for Math.
- Further analysis is needed to clarify whether the complete package of inputs to the FTS model produces learning outcome results that are better than those in the non-FTS of various types, as there are many variables involved.
Improve the quality of preschool and primary education (ages 4 to 11) with a focus on the socio-economically disadvantaged and very disadvantaged contexts: Modest
- Equipment for 868 FTS classrooms was purchased. This was slightly above the 2012 target of 850, and above the original target of 694 classrooms. 175 FTS received computers and equipment,
- Video equipment was provided to 34 FTS,
- 280 sets of learning materials were distributed to FTS and Priority Attention Schools (PAS),
- 40 sets of bilingual materials and one set of library books were procured for each FTS,
- One textbook per child in 1st and 2nd grades was produced and distributed.
- 72 FTS and PAS schools received inputs to strengthen links between schools and communities, including social supports for children experiencing problems.
- Compensation was increased for 60 teachers who were reclassified (the ICR does not state how they were reclassified) in 2006.
- A new teacher training institute was constructed, and books and new teacher training materials were provided to all teacher training institutes.
- A new pre-service teacher training curriculum was designed.
- 2,785 education personnel were trained in the FTS model, and 2,100 were trained in PAS. 415 teachers were trained in bilingual education.
- An in-service training system was developed.
- There was national participation in international assessments (Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study).
- School construction was developed in an innovative fashion, with input from the communities resulting in innovative school designs adapted to the local environment. This was designed to support the overall goal of community engagement in education.
- The percentage of sixth grade students from disadvantaged and very disadvantaged backgrounds who performed at a satisfactory level changed between 1999 and 2009 as follows: in FTS only, increased from 53.8% to 64.8% for Language, but decreased from 35.6% to 33.4% for Math. For all disadvantaged/very disadvantaged students (FTS plus non-FTS), the performance was better: an increase from 48.8% to 63.1% for Language, and an increase from 27.9% to 31.8% in Math. Some of these results may not be statistically significant.
- The data presented on achievement for students from disadvantaged backgrounds and on the equity gap run from 1999 through 2009, while the project's time period was 2002 through 2012. It is therefore uncertain that the outcomes presented here can be attributed with confidence to the project's interventions. .The Region provided additional information on repetition rates pointing to the majority of improvements occurring under the project period. No learning assessment has been held since 2009.
- It may require more time for the project's inputs to fully impact disadvantaged student populations in the FTS system. There are no data in the project for other issues the FTS system is designed to address, such as violence in urban areas.
Increase the efficiency of the preschool and primary education system with a focus on socio-economically and disadvantaged and very disadvantaged contexts: Substantial
- The achievement of this PDO is linked to outputs listed under the previous two objectives. Improvements in access and better staffed and equipped schools would have affected retention and completion rates for the disadvantaged. The Education Improvement Projects increased community engagement in schools, a known factor in creating a supportive and more efficient learning environment.
- 320 computers and 120 printers were provided, and 2,600 personnel were trained in data collection and the use of equipment.
- 912 Education Improvement Projects (PMEs) were developed (not meeting the target of 1,100 PME). This activity was integrated into the regular education system program.
- The Ministry of Education now has a functioning data collection system able to provide national level data for the education system and the project, although more needs to be done to effectively measure learning achievements.
- Repetition rates improved, decreasing in the FTS for the lowest two income quintiles for first grade from 23.2% in 1999 to 12.8% in 2012, surpassing the target of 15%; and for second grade from 17.1% in 1999 to 6.1% in 2012, surpassing the target of 10%. For all schools (FTS plus non-FTS), there was also improvement, but not as strong as for the FTS alone: a decrease in repetition of first grade from 23.4% to 18.5% from 1999 to 2012, and a decrease in repetition of second grade from 17.6% to 9.7% over the same time period. Data presented by the Region demonstrate that the majority of these improvements took place between 2002 and 2009. These improvements can partly be attributed to the effect of improved pre-school access and quality.
- Drop-out rates for children in grades one through six in the first and second income quintiles decreased from 2.1% in 1991 to 1.0% in 2012 for the FTS alone, and from 2.2% to 1.5% in all (FTS plus non-FTS) schools.
- Poor attendance rates for children in grades one through six in the first and second income quintiles decreased from 9.8% in 1999 to 8.3% in 2012 for the FTS alone, and from 10.0% to 9.8% in all (FTS plus non-FTS) schools.
Efficiency is rated Modest. The Economic Analysis in the Project Appraisal Document calculated a possible Economic Rate of Return of 37% (with the minimal return being above 13%) on activities in Component 1 of the project, which focused on expanding the FTS model (largely construction). This seems high. The Cost Benefit Analysis carried out for the PAD noted the possibility of increased indirect costs over time due to the expanded salaries and infrastructure created for the FTS. The Economic Analysis noted two major benefits from the expansion of the FTS model, covering the period 2002-2035:
1. Students would benefit from improvements in the quality of their education which would translate into better qualifications and improved earnings in the labor market. Over the long term this should result in an increase of human capital per year of schooling and a more efficient use of education funds.
2. In addition the FTS model had an impact on the labor force participation of primary caregivers who could work longer hours and contribute to their income and to the economy overall.
When recalculated at the time of the 2009 Additional Financing request, the ERR was estimated to be 15%,with additional financing, taking into account a lower number of beneficiaries had originally been anticipated. The ICR bases it's overall economic analysis on that carried out for appraisal of the follow-on operation. This gives an ERR of between 7.4% and 9.6%. It is not entirely clear how this analysis is accurate for the project under review; although they both support the FTS model, project components are not the same. The ICR also undertakes a financial analysis which notes that higher education costs as a result of the increases in salaries and training have been financed by increases in the education budget. It makes an assumption that this will not be a burden on the State budget, as average per-student spending in primary and secondary schools is 7.2% as opposed to a regional average of 13.1%. This is not a valid assumption, as there are many variables between countries and systems. The ICR calculates an IERR between 7.3% and 9.6% based on assumptions of improvements in lifetime incomes for students.
Construction costs under the project rose on average by about 144% during the period 2002-2009. The Uruguayan National Statistical Institute Construction Cost Index reported an increase in construction costs from August 2006 to June 2009 of about 27% (Project Paper on Additional Financing, para.13, page 3 ). This resulted in considerably higher than anticipated overall project costs. This was partly because the ratio between rehabilitation and construction costs was reversed (70/30 to 35/65%) by the end of the project. Additional information provided by the Region reinforces these points, also noting that the decision to shift from classroom rehabilitation to new construction meant that there was a long-term benefit to the system in terms of the efficiency of new buildings and lower maintenance requirements in the new schools.
NOTE : The ICR estimate below is based on the estimated coverage at appraisal for Component 1 plus the additional financing.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated