|1. Project Data:
ICR Review Date Posted:
|Education System Realignment & Strengthening Program (apl #2)
Project Costs(US $M)
Loan/Credit (US $M)
Cofinancing (US $M)
|Japan Ministry of Finance- Population and Human Resource Development Grant (PHRD)
Board Approval Date
|Central government administration (50%), Secondary education (20%), Primary education (20%), Tertiary education (10%)|
|Education for all (50% - P)
Law reform (25% - S)
Education for the knowledge economy (25% - S)|
||ICR Review Coordinator:
|Susan Ann Caceres
||Judyth L. Twigg
|2. Project Objectives and Components:|
a. Objectives:The loan was the second phase of a planned three-phase Adaptable Program Loan. The Program Objective was to "effectively realign the objectives of the education system and enhance the policy and the management capacity to improve efficiency, quality, and relevance of learning outcomes. The overarching objective was to better prepare students to meet the demands of a market economy and a democratic society" (Project Appraisal Document, p. 4).
According to the Project Appraisal Document, the Development Objective of the project was " to increase the quality and efficiency of primary and secondary education in Georgia through the implementation of the institutional and policy framework developed under APL 1, as well as the reconstruction of schools in dire physical condition" (p. vi). The same objective was stated in the Financing Agreement (p. 4).
b. Were the project objectives/key associated outcome targets revised during implementation?
c. Components:The project had three components:
1. Improving the Pedagogical Environment (appraisal, US$ 7.2 million; actual, US $5.23 million) was to support changes in curriculum content, teaching methodology, student assessment, initial teacher education, and continued teacher professional development, building on the institutional and policy frameworks supported by the previous loan. It contained two subcomponents:
- National Curriculum and Student Assessment System Development (appraisal, US$ 5.58 million; actual, US$ 4.13 million) was to improve teaching methodologies in line with the national curriculum framework in almost all grades and subjects, establish a system of student assessment for 4th grade, and assist Initial Teacher Education institutions to adapt their programs to changing standards, curriculum, and student assessment methodologies.
- Professional Development of Teachers (appraisal, US$ 1.62 million; actual, US$1.10 million) was to improve the qualifications of an unspecified number of teachers through the development and implementation of a teacher professional development system.
2. Improving the Physical Learning Environment (appraisal, US$ 13.95 million; actual, US$ 17.09 million) was to improve the physical learning environment in schools in emergency (delapidated) conditions to complement investments in improving the quality of the pedagogical learning environment and adoption of more efficient building standards and school consolidations.
3. Project Management, Monitoring and Evaluation (appraisal, US$ 2.58 million; actual, US$ 1.53 million) was to provide institutional support to the Ministry of Education and Science, the Municipal Development Fund (MDF), the National Curriculum and Assessment Center (NCAC), and the Teacher Professional Development Center (TPDC) related to project operations, audits, and project monitoring and evaluation.
d. Comments on Project Cost, Financing, Borrower Contribution, and DatesProject Cost: There was significant exchange rate fluctuation between the local currency, the US dollar, and the Special Drawing Rights (SDR). As a result, the project gained approximately US$ 1.1 million, which was used to cover increases in construction costs. An undisbursed balance of US$ 9,458 was canceled after the end of the grace period for reconciliation of accounting (ICR, p. 12). Project costs were higher than anticipated for the second component, and lower than planned for the other components. After the August 2008 conflict between Georgia and Russia, there was more demand for construction and so costs increased for the civil works financed by the second component (ICR, p. 10).
Financing: The project was financed by an International Development Association Credit in the amount of US$ 15.84 million equivalent and complemented by a Japanese Population and Human Resource Development (PHRD) grant in the amount of US$ 4.95 million (ICR, p. 29), which was used to support in-service teacher training, curriculum reforms, and textbook translation (ICR, p. viii).
Borrower Contribution: Government funding was anticipated to be US$ 3.78 million, but was actually US$ 3.06 million (ICR, p. 29).
Dates: The project closed 20 months later than initially planned. The first extension was granted on November 17, 2009 to change the closing date from December 31, 2009 to December 31, 2010 to complete civil works underway and to support the Borrower with key education reform activities related to in-service teacher training, curriculum reforms, and textbook translations into minority languages. The second extension was granted on December 16, 2010 to revise the closing date from December 31, 2010 to August 31, 2011 to complete activities under the PHRD Grant.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:Objectives: High
Given the low funding that Georgia devoted to education in comparison to other similar countries, the education sector was faced with constraints at the time of project preparation: inadequate physical infrastructure with major deterioration in the physical conditions of many schools and the need to modernize the curricula and teaching methods, while at the same time trying to use resources more efficiently by increasing the student-teacher ratio, given the declining school age population in the country.
Education was an essential pillar of the Georgian development strategy, Economic Development and Poverty Reduction Program (2006). The focus of education reform was to bring the Georgian education system in line with European standards by introducing a national system for assessment and examinations. The Bank's Country Partnership Strategy (2010-2013, current at project closure) focuses on improving efficiency of use of public resources and the quality of education. The project's objectives were highly relevant to the Country Partnership Strategy and to the goals of the Government's strategy, as the project aimed to establish an outcomes-based national curriculum framework, invest in the in-service training of teachers to upgrade their pedagogical skills, design and implement a per-student financing system, and begin plans for better school optimization, all of which would address the key challenges in the sector. The government's current Strategy for Development of Education (2010-2015) contains initiatives reflective of project objectives (e.g. improvement of quality), which also demonstrates their continued relevance.
b. Relevance of Design:Design: Modest
The activities outlined in the project's components could be expected to lead to the intended outcome of improved quality, since there was a logical causal chain between that objective and the activities intended to achieve it (PAD, pp. 31-33). However, there was a weak logical chain relating the efficiency objective to associated components/activities. As noted by the ICR (p. 8-9), the design did not address the underlying causes of the increasing student-teacher ratio or explore other solutions to increase school efficiency. As well, there were insufficient project activities (and inadequacies in the case of the per student funding formula) to lead to increased system efficiency, as well as increased student-teacher ratio.
|4. Achievement of Objectives (Efficacy) :|
Increase the quality of primary and secondary education in Georgia: Substantial
Outputs related to teacher professional development:
- Guidelines and standards were developed for the accreditation of teachers' continued professional development programs.
- 172 Continuing Professional Development Programs were accredited by the Ministry, and training was outsourced to private providers. The ICR (p. 16) notes that this program was discontinued in 2010, when the quality of some of the accredited programs was questioned and responsibility for training was recentralized at the TPDC.
- 25,000 primary and secondary teachers and 2,000 school principals were trained in the new national curriculum. According to the project team, when further revisions were made to the national curriculum, additional training was then provided to 8,000 teachers.
- Professional development was provided for National Curriculum and Assessment Center core staff in the areas of curriculum reform and national assessment.
- 10 videos on student-centered teaching were developed by the National Curriculum and Assessment Center, and monthly bulletins on this subject were provided to all schools. The ICR does not specify how many teachers were exposed to these videos.
- 67 books/literature to support professional development were translated from English to Georgian.
- Teacher Professional Standards were developed and finalized (Code of Ethics, Second Language Standard, Special Needs Teacher, Vocational Teacher, School Principal). The ICR does not state whether and how these standards were implemented.
- Professional teacher resource materials were developed, such as a three-volume manual for teachers and videos on teaching methods.
- 2,163 teachers used government-provided vouchers to participate in a literacy training program that they selected from private providers, and 239 teachers used the voucher program to participate in a Georgian as a Second Language (GSL) program by the end of 2010. This voucher program was discontinued during the project, given the lack of monitoring of the private providers. By the end of the project, training was provided by the government's Teacher Professional Development Center.
- A Teacher Induction Pilot Program was established, providing support for 48 inductees and 28 mentors between 2008 and 2009. According to the project team, in 2010 this program was continued and expanded by the government through budget support.
Outputs related to curriculum and teacher certification standards:
- The new national curriculum developed under the first phase of the project was piloted and then implemented in all schools across the country, except for grade six in Georgian schools and grades five and six in minority schools.
- New textbooks were published, and these new textbooks were translated into minority languages (e.g. Russian, Armenian, and Azeri). The ICR did not provide the number of textbooks developed and distributed. However, the project team provided a list of 52 titles covering grades one through twelve.
- 167 new textbooks were accredited by the National Curriculum and Assessment Center.
- 90% of children were taught the improved national curriculum, which exceeded the target of 70% of students.
- New teacher certification standards were developed, based on the new national curriculum framework. The ICR does not state whether or how these standards were implemented.
- A Teacher Certification Charter was adopted by Ministry decree No. 1101 in December 2009.
- A database of registration for teacher certification examination was established.
Outputs related to civil works:
- 38 schools identified in emergency conditions were rehabilitated and refurnished and found to be of satisfactory quality. These schools supported 3,066 students (ICR, p. 36).
Outputs related to improving education quality:
- National assessments in Georgian and Mathematics for grade 4 and 9 students were conducted. The assessment was based on a sample of students from schools across Georgia. Both assessment reports are publicly available on the National Curriculum and Assessment Center website, but the ICR did not provide this data.
- 43 teachers participated in the Teach for Georgia Pilot Program, which was designed to attract and retain teachers in remote areas. According to the project team, of the total number of participants, 41 remained for the first year of teaching, 32 continued for the second year, and 16 remained for the third year.
- 71% of Georgian teachers were able to apply at least one active instructional methodology, which attained the target of 70% and was an increase from the baseline of 10%. (Source: Classroom Observation Research, ICR, p.14 and PAD, p. 34). The ICR (p. 16) also reported survey data showing more positive attitudes by more teachers towards more active teaching practices and student engagement in comparison to a previous survey conducted in 2007.
- 14.8% of teachers who took the teacher certification examination passed and were certified (or 6.4% of the total teaching force), which did not meet the target of 20% of teachers certified according to the new standards. The ICR (p. 18 and p. 55) provides two reasons why this target was not met. A relatively low number of teachers participated in the exam, since the new certification requirement will not become mandatory until 2014, and the implementation of the exam was late and only began in 2010. Of those that participated in the examinations, 85% did not pass both required tests.
- There was a seven-point increase in 4th grade Mathematics and Georgian scores between 2004 and 2010 (ICR, p. 22). The ICR does not present the exact data from these assessments, but it indicates (p 22) that there were comparison and attribution issues.
- The ICR (p. 31) reported that assessments of ninth grade students were conducted in 2009 and 2010 and that these results were publicly available on the NCAC's website. However, the results of these assessments are not provided in the ICR.
The ICR does not discuss other factors outside the project that could also have contributed to improving education quality.
Increase the efficiency of primary and secondary education in Georgia: Modest
- Revisions to the per student funding formula, which was designed during the first phase of the APL, were made to better account for the cost of operating small rural schools.
- 100% of schools are in charge of their maintenance budgets.
- Several manuals, Improving Schools' Physical Learning Environment, Human Resource Management Manual, and School Sanitation Maintenance, were published and distributed by the Ministry of Education and Science, and 500 school principals were trained on their content. Since the funding formula financed routine school maintenance, these documents supported new approaches to school maintenance planning and management to better sustain the physical environment.
- A teacher vacancy website was designed to post regular information on existing teacher vacancies across the country to support easy access to this information.
- The number of schools carrying a deficit decreased from 1,400 schools in 2010 to 167 schools in 2011. This means that 93% of schools are managing their resources without any deficits. These remaining schools either have more than one school building under their jurisdictions or more than one language of instruction, which increases their operating costs (ICR, p. 19). While all schools had autonomy in the management of resources through the per student funding formula, this new mechanism did not become the sole source of public funding for schools, which did not attain the target of 100% of schools having autonomy in managing resources related to the teaching and learning process through the use of per-student financing as the sole source of public funding and accountable to the appropriate authorities in the use of those resources.
- The project aimed to increase efficiency by raising the student-teacher ratio. In fact, it decreased from 12:1 in 2006 to 10:1 in 2011, which did not meet the target of increasing to 16:1 and was a deterioration from baseline. The ICR (p. 19) indicates that the school consolidation undertaken by the government was not sufficient to increase the national average student-teacher ratio.
The Government did not elaborate a strategy to increase the student-teacher ratio based on the specific situations of urban, rural, and mountain areas, which impacted the ability to increase the efficiency of primary and secondary education. As well, the revisions made to the funding formula during this project did not adequately account for schools having more than one school building under their jurisdictions or more than one language of instruction, which increases their operating costs (ICR, p. 19).
Program Objective: Effectively realign the objectives of the education system and enhance the policy and the management capacity to improve efficiency, quality, and relevance of learning outcomes to prepare them to meet the demands of a market economy and a democratic society.
By the end of the operation (the second phase of the overall program), there were improvements in education quality and relevance of the learning with the development of a new national curriculum and training of 25,000 teacher and 2,000 school principals, and conducting of national assessments. Limited progress is evident in improving efficiency in the system. There is no indication that students are better prepared to meet the demands of a market economy or democratic society.
The PAD did not conduct an economic analysis, but justified the project interventions on the basis that poor school infrastructure negatively impacts learning and performance, since lack of heating constitutes the main reason for long-term absenteeism in Georgia (PAD, p. 65). The PAD (p. 66) reported that 9.03% of all school buildings in Georgia were in classified as in emergency conditions. The PAD (p. 68) noted that the educational investments of this project were justified, since they would mainly benefited children from poor households.
The ICR did not conduct an economic analysis, nor did it provide unit cost comparisons. However, information contained in the ICR and in the Government’s reply indicated substantial efficiency. First, the cost of project management was 41% lower than the appraised estimate (ICR, p. 29). The ICR states that implementation went smoothly and that fiduciary obligations were completed satisfactorily, suggesting that the Implementing Agency was able to use fewer resources than planned to manage and monitor the project. Second, 38 schools in emergency condition were rehabilitated and refurnished and seven schools fully reconstructed (ICR p. 18). This was accomplished despite the escalating cost of materials as a result of the increased demand for construction materials with the need for post-conflict reconstruction (ICR, p. 10). Third, the new schools could promote efficiency gains, since each school was consolidated with two or more other existing schools, since the Ministry provided a school bus to facilitate this process (ICR p. 18). The new schools offer space that could be used for extracurricular activities or vocational programs, if the Government encourages effective utilization of the buildings. Fourth, maintenance costs and implications for the recurrent budget of the education sector have been factored in the per student financing formula, so that schools have an adequate level of resources with the most recent revision of the per student financing formula (ICR p. 13). 93% of all schools were able to manage their resources without any deficit (ICR p. 22).
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated