|1. Project Data:
ICR Review Date Posted:
|Kh--education For All Fast Track Initiative Catalytic Trust Fund
Project Costs(US $M)
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|Primary education (71%), Pre-primary education (22%), Central government administration (5%), Sub-national government administration (2%)|
|Education for all (100% - P)|
||ICR Review Coordinator:
||Judyth L. Twigg
||Christopher D. Gerrard
|2. Project Objectives and Components:|
This project represents financing from the Education-For-All Fast Track Initiative (FTI) Catalytic Fund for the Cambodia Education Sector Support Scale Up Action Program.
The stated Program Development Objectives as found in the Grant Agreement and the main body of the Project Appraisal Document (PAD) were succinct:
· To speed up progress towards improved grade 1 right-age entry, net enrollment, retention, progression rates and grade 6 completion rates in primary education.
A different version of the objectives was indicated in the Results Framework of the PAD. For purposes of the ICR and this ICR Review, the PDOs above will be taken as relevant and binding.
b. Were the project objectives/key associated outcome targets revised during implementation?
1: Expanding Early Childhood Education (ECE) (Appraisal: US$4.7 million, Actual: $4.09 million) included three subcomponents:
(i) ECE Facilities Expansion (appraised at US$2.6 million or 4.5 percent of total) aimed to expand and update the stock of ECE classrooms at 650 primary schools and 25 new buildings, as well as expand the annual intake of the Preschool Teacher Training College to 200 teacher trainees.
(ii) Home- and Community-Based ECE Expansion (appraised at US$1.2 million or 2.0 percent of total) was intended to launch 450 new community preschools and 450 home-based preschools.
(iii) ECE Development Policy and Capacity Building (appraised at US$0.9 million or 1.6 percent of total) aimed to update the ECE strategy and conduct community sensitization activities as well as provide regular in-service pedagogical support and supervisory services.
Overall, this component aimed to increase the number of young Cambodians in disadvantaged communities who could benefit from ECE services through the construction of new formal preschool classrooms as well as the development of new community-based and home-based ECE interventions.
2: Improving Primary Education Access and Quality (Appraisal: US$50.7 million, Actual: $47.86 million) included three subcomponents:
(i) Expanding Primary Education Facilities (appraised at US$39.9 million or 69.5 percent of total) to construct 2600 classrooms to complete 650 incomplete primary schools.
(ii) Stimulating Demand for Education (appraised at US$1.9 million or 0.3 percent of total) was to implement a national publicity campaign on right-age entry and scholarships for select grade 3 students.
(iii) Support to Child Friendly Schooling (appraised at US$6.3 million or 10.9 percent of total) aimed to build local capacity to improve school effectiveness, governance, and results by encouraging right-age enrollment, retention, transition to lower secondary education, and improved learning. This was to be accomplished through finalization and roll-out of a school self-assessment tool, in-service training on child-friendly methods, and leadership training for principals.
(iv) School Improvement Grants (appraised at US$1.5 million or 2.6 percent of total) were intended to supplement school operating budgets in areas identified by the self-assessment tool.
(v) Instructional Materials Provision (appraised at US$1.0 million or 1.8 percent of total) was to include the establishment of libraries with age-appropriate library books.
Overall, this component aimed to promote the national goal of universal primary completion through the construction of new primary classrooms and provision of demand-side incentives, while promoting child-friendly schooling initiatives to enhance educational quality and strengthen teacher professional capacity.
3: Promoting Institutional Development and Capacity Building (Appraisal: US$2.0 million, Actual: 4.56 million) consisted of two subcomponents:
(i) District Offices of Education Facilities Expansion (appraised at US$0.9 million or 1.6 percent of total) was to build 60 new District Offices of Education buildings, inclusive of furniture, in rural areas where there had been none.
(ii) Financial Management, Procurement and Program Management Capacity Building (appraised at US$1.1
million or 1.9 percent of total) was to strengthen internal and external audit, accounting, reporting, transparency, and decentralization of procurement. Capacity building was to occur with select province-level staff on accounting, record keeping, and financial monitoring. A client construction unit was also to be established to monitor the public works construction of the project.
Overall, this component was intended to enable decentralized education management and monitoring by Departmental and Provincial Offices of Education and the Inspectorate of Education through the rollout of a new school inspection policy, geared towards enhancing direct support services and responsiveness to school needs, and through the strengthening of administrative processes for procurement and financial management at all administrative levels.
An additional sub-component of an Early Grade Reading Assessment was added to Component 2 during the project’s third restructuring in December 2010. These assessments were to be carried out in grades 1 and 2 and were requested by the Ministry of Education Youth and Sports (MoEYS) following a World bank workshop on early grade reading.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
The original total commitment was US$57.4 million, and the project stayed at that amount despite being restructured three times. Differences in disbursement amounts reported in the ICR of $57.36 million (p. i) and $56.51 million (p. 26) are due to the former not accounting for the 4-month grace period in the Bank disbursement letter.
Significant deviations in expenditure -- lower than planned for Components 1 and 2, and higher than planned for Component 3 -- are attributable to several issues, explained by the project team. First, construction costs rose by 50-60% over the project period (ICR, p. 7). Second, because of those higher costs, additional classrooms in Component 2 were done as rehabilitations or add-ons to existing schools rather than new free -standing schools. No such reduction in costs by adding onto existing structures was possible for the construction of the district offices specified in Component 3. Third, a salary supplement for a wage increase at a rate harmonized across all donors, termed Priority Operating Costs, was added in the second restructuring, and its costs accrued largely to the Client Construction unit in Component 3. Finally, nearly twice as many trainings and workshops as planned at appraisal were accomplished, most of which accrued to Component 3.
The Fast Track Initiative Catalytic Fund provided US$57.4 million in grants. Actual disbursement was US$57.36 million. The ICR is unclear regarding the remaining US$40,000. No amount was cancelled during implementation. With the revised start dates and delays, the initial disbursement amount of US$18 million planned for 2008 Q4 was not realized until 30 months later in 2011 Q2; from that point, however, disbursement followed the slope of the anticipated trajectory fairly closely.
There was no planned Borrower contribution.
- The project was approved on 10 April 2008, and the effectiveness period began 13 June 2008.
- The project was restructured three times, resulting in the extension of the project closing date from 30 June 2010 by a total of 30 months to 31 December 2012.
- 27 April 2010: A 24-month extension to the original project closing date, from 30 June 2010 to 30 June 2012, took place because the General Election prevented the Ministry from appointing key project personnel in a timely fashion and because of delays in completing construction activities.
- 19 December 2010: A restructuring replaced planned salary supplement and incentive schemes with the Priority Operating Costs scheme and instituted the Early Grade Reading Assessment sub-component.
- 28 June 2012: A 6-month delay in project closing, from 30 June 2012 to 31 December 2012, was approved to recover lost time due to flooding and to repair damage to project infrastructure caused by the flooding.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:
High. The most recent Cambodia Country Assistance Strategy (2005-2008) includes two pillars: removing government constraints to attaining the Cambodia Millennium Development Goals (CMDGs), and supporting the formulation and implementation of a government-led strategy and investment program for achieving the CMDGs. While the Project Development Objectives did not explicitly address the first pillar, the way the project was implemented -- through the Government’s institutions, most notably the Ministry of Education, Youth, and Sports -- is certainly relevant in improving government capacity, which can in turn reduce governance constraints.
The second pillar focuses on two objectives: supporting a government-led strategy to achieve the CMDGs, and supporting sectors and sub-sectors critical to achieving the CMDGs. The project is more directly aligned in its objectives and design to this second pillar, with respect to the second Cambodian Millennium Development Goal to achieve universal nine-year basic education. This CMDG has 10 targets covering admission, enrollment, survival (retention/progression), and literacy rates as well as the proportion of children out of school and gender ratio disparities. Of the five main objectives listed for the project, four are directly related to these ten CMDG indicators.
At the time of appraisal, the education system in Cambodia had made great strides in attendance (91.3% in 2006) but was still deficient in several important areas. While the correct age for entry into primary school was six years, Cambodian children entered quite late, at age 7.8 years on average. A gender disparity in secondary graduates was attributed to differential progression rates from early in the education cycle. Moreover, primary completion rates of 85% were even lower for poor children. Repetition rates had seen significant progress in the decade prior to the project, dropping from 26.3 to 11 percent. The pupil:teacher ratio was stagnant and high at around 50, and there was real concern over low teacher capacity. Learning outcomes were poor as measured by student knowledge of the official Khmer and mathematics curriculum. Finally, while the government strategy called for a move toward decentralization, local capacity for leading and improving education was weak. The project was therefore highly relevant to country conditions at the time of appraisal.
b. Relevance of Design:
Modest. The project’s design (its planned activities and policy areas) is consistent with its stated objectives and is well-informed by solid research on both the local Cambodian context and global best practice. Still, the Results Framework of the PAD is under-developed and inaccurate. The design also did not sufficiently take into account capacity constraints of the Ministry of Education, Youth, and Sports (MoEYS).
The project’s three components of early childhood education, improving primary education access and quality, and promoting institutional development and capacity building reflect well the PDOs of the project. The first two subcomponents directly link to the five outcomes of right age of entry at grade 1, net enrollment, retention, progression, and completion. The third subcomponent of promoting institutional development and capacity building within the Ministry of Education is less directly related to the outcomes, but is not orthogonal.
Clearly the design of the first two components in particular benefitted from a careful and thorough understanding of the best available evidence on early childhood education and primary school attendance. It relied heavily on both solid impact evaluation evidence from other contexts and data and analysis specific to Cambodia.
However, while the design addressed demand-side constraints to primary education with its scholarship program, it did not consider the demand constraints (in addition to the planned-for supply constraints) that inhibit families from taking advantage of early childhood education (ECE) programs. Moreover, the design did not consider that the (apparently not uncommon) current practice of children beginning to attend primary school a year early at age 5 would have positive benefits for the child over and above what an ECE curriculum could provide, thus lowering children’s cognitive development by taking them out of primary school and putting them in ECE interventions. Similarly, the design (and the ICR) also did not consider the effects and interactions of other Cambodian programs contemporary to the Education Sector Support Scale Up Action Program (ESSSUAP).
Finally, the design did not account for exogenous factors that affected the implementation of the project. As discussed in the ICR, the design gives significant ownership to, and so places a heavy burden on, the Government (specifically the MoEYS) and contractors to detail and carry out the planned activities. The 24-month delay in implementation was due largely to issues around a General Election that could have been taken into account in the design of the project.
|4. Achievement of Objectives (Efficacy) :|
Objective 1: To speed up progress towards improved grade 1 right-age entry. Modest.
The project did not include an indicator for Grade 1 right-age entry in its monitoring plan. However, several program indicators hint that the level of grade 1 right-age entry may have improved under the implicit program logic that primary school age-at-entry can be changed through early childhood education. ECE enrollment of 5 year olds and 3-5 year olds both increased and essentially met their targets. Primary school net admission rates also increased from a high base. Whether these improvements were increases over the previous trend (e.g. “speed up progress”) is also not known, as only 1 baseline period is given in the ICR.
The ICR also does not discuss other factors that may have led to the outcomes reported below; indeed, the ICR points out that while these are national-level trends, students within the project area accounted for no more than 6% of the total student population.
- The project built 424 early childhood education classrooms of the targeted 675, achieving 62.8% of the original target value.
- The project launched all of the targeted 450 community preschools.
- The project also supported 450 new home-based education programs, meeting the target of 450.
· Early childhood education enrollment of 5-year olds increased nationally from 31.4% at the end of CY 2008 to 52.66% at the end of CY 2012, exceeding the 50% target for the program.
· Early childhood education enrollment of 3-5-year olds increased nationally from 15.57% at the end of CY 2008 to 28.89% at the end of CY 2012, below the 30% target for the program.
· Net admission rates (age group not given, though grade 1 is indicated) increased nationally from 89.5% at the end of CY 2008 to 94.1% at the end of CY 2012, not reaching the target of 100%. The high base of admissions (nearly 90% at baseline) suggests that getting children into school is not as much of a challenge as students entering at the right age. No age data are given for those entering grade 1.
· Though the monitoring system did not collect data on enrollment age, an impact evaluation (Bouguen et al 2013) on the project presents convincing evidence that the program was indeed responsible for increasing the number of children who entered primary school at the right age. Unfortunately, it did so at the cost of lower cognitive attainment among recipient children. A large portion of the children in the relevant counterfactual group attended primary school a year early—at age five—and so received greater cognitive stimulus at primary school than those in the Bank-funded ECE program.
Objective 2: To speed up progress towards improved net enrollment. Substantial.
Though the project did not reach the target of 100% national enrollment, it was unlikely to do so working with 6% of the student population. Still, overall enrollment did increase, and two well-constructed impact evaluations demonstrate that the scholarship program improved attendance, a distinct though related construct to enrollment. Students receiving program scholarships were more likely to be in school and attended more hours of school.
· The project awarded 21,780 scholarships, or 91.12% of the targeted 24,000 scholarships. These scholarships were awarded conditional on enrollment.
· The net primary enrollment rate increased nationally from 93.3% at the end of CY 2008 to 96.4% at the end of CY 2012, 3.6 percentage points below the 100% target for the program and covering 46.27% of the gap to the target.
· An impact evaluation of the scholarship program by Barrera-Osorio and Filmer (2013) asserts that both the merit-based and the means-based scholarships increased “enrollment and attendance” over equally qualified candidates randomly assigned to a control group. The impact evaluation uses three proxies for enrollment and attendance, including “the proportion of students reaching 6th grade” and “the highest grade completed” (p. 14). The paper notes that “the intervention is aimed at directly incentivizing higher enrollment and attendance rates, so scholarship students must stay enrolled… until they graduate from primary school (6th grade)” (p. 14). Recalling that the problem of net enrollment in Cambodia is far more concentrated in the later grades than in the earlier grades (only 60% of students in the control group made it to sixth grade or beyond, compared to a 93% net primary enrollment rate nationally), the scholarship program, which began in the middle grades and resulted in a large and highly significant increase in the proportion of students reaching grade six of 18 and 13 percentage points for the poverty and merit-based programs, respectively, is highly plausible to have substantially sped up progress towards improved net enrollment among those in the project’s area. The scholarship program is also similar in design and effect to a Cambodian Lower Secondary School scholarship program that increased enrollment on the order of 20-25 percentage points (Filmer and Schady, 2008, 2011).
Objective 3: To speed up progress towards improved retention. Substantial.
Retention is interpreted in the PAD as “retention of students in upper primary” (p. 23). Though the program did not collect indicators or monitoring and evaluation data on retention, evidence from a well-executed impact evaluation gives strong support that the program caused better attendance among 6th graders, a likely close correlate to retention.
· In addition to the Output of Objective 2, the project completed 424 of formerly “incomplete” primary schools, reaching 2/3 of the target of 650 schools. The ICR partially attributes this shortfall to a change in tactics to also include construction (expansion) of overly crowded schools. Increasing the number of schools with upper primary schools by completing incomplete schools is presumed to have an effect on improving retention / enrollment / attendance of upper primary grade students.
· The Barrera-Osorio and Filmer (2013) impact evaluation estimates that the poverty-targeted scholarships of the program increased the probability of reaching 6th grade by 18 percentage points.
· The Barrera-Osorio and Filmer (2013) impact evaluation estimates that the poverty-targeted scholarships of the program caused students to complete 0.33 more grades.
Because retention is a necessary condition for increasing the number of grades completed by a population, by substantially increasing grades completed (and the likelihood of reaching 6th grade), the scholarship component of the program very plausibly improved retention of students in upper primary.
Objective 4: To speed up progress towards improved progression rates. Substantial.
“Progression rates” is not further defined in the policy documents, but may reasonably be interpreted as moving along in grades completed and avoiding repetition. The effort to increase the number of complete primary schools is implied to improve progression by providing a class for students to progress into. Also, the inferred (though unstated) program logic of the scholarships, especially merit-based scholarships, is that rewarding students for attending may help them progress in their education.
· The relevant outputs are the same as those in Objective 3.
· The program had a target indicator of reducing grade 1 repetition from 18.7% to 4%. Grade 1 repetition rates actually decreased to 10.9% nationally. It is unclear how much of this decline is attributable to the program, given the low share of students nationally who were involved in the program.
· The Barrera-Osorio and Filmer (2013) impact evaluation estimates that the merit-targeted scholarships of the program increased the probability of reaching 6th grade by 13 percentage points.
· The Barrera-Osorio and Filmer (2013) impact evaluation estimates that the merit-targeted scholarships of the program caused students to complete 0.2 more grades than those in the control group.
Objective 5: To speed up progress towards improved grade 6 completion rates. Modest.
The program logic of grade 6 completion, as distinct from the other objectives, is not clear. The project design seems to imply that building more classroom capacity and an increase in the number of “complete” primary schools. Similarly, providing scholarships may have induced students to complete the sixth grade. The indicators for this outcome are taken from national-level data, and the influence of the Bank project, given its low number of enrolled students, is not clear.
· The relevant, directly related outputs are the same as those listed in Objectives 3 and 4.
- The project’s indicator for this objective, primary completion rate, increased from 86.25% at the end of CY 2008 to 89.7% at the end of CY 2012, well short of the target of 95%. The increase in primary completion rates of less than 3.5 percentage points covered only one third of the intended distance from baseline to target.
- The Barrera-Osorio and Filmer (2013) Impact evaluations established that the scholarship programs increased the number of grades completed and the probability of reaching grade 6, which are necessary but not sufficient conditions for increasing grade 6 completion rates. Because the scholarships increased the probability of reaching grade 6 and the total number of years completed, it may be that the program increased grade 6 completion rates. However, this is not clearly established.
Modest. The project was not subjected to an analysis of economic efficiency at appraisal. However, the ICR conducts a cost-benefit analysis of Components 1.1 and 1.2 on Early Childhood Education (facility and home- and community-based ECE expansion) and Component 2.2 on stimulating demand for education (the scholarship program). Together these components account for US$5.7 million, or 10% of the total value of the project. The internal rates of return for all of the evaluated components are above the presumed opportunity cost of capital (10%). This indicates substantial levels of efficiency, and reasonable confidence around those estimates, for these components.
Point Value (Range) Coverage / Scope
ECE Facilities, Home, Cmty Expansion
15.48% (14.77 - 18.31%)
Stimulating Ed Demand (Scholarships)
10.89% (8.1 – 13.88%)
The quality of the ICR's economic analysis itself is quite high. It does commendable work in clearly laying out the assumptions of the analysis and attempting to make a full accounting for both direct and indirect costs, using estimates of the benefits based on sound empirical studies, making reasonable projections, and conducting sensitivity analysis.
The large enrollment and attendance effects of the CCT scholarship program were attained at an extremely small cost (US $20 per year) when compared with the more modest gains and significantly higher costs of similar programs worldwide (Fiszbein and Schady, 2011; Filmer and Schady 2010).
Still, the ICR seems to indicate that the 25-year time horizon does not include the two-year implementation delay as it should. And while it cites an empirical study on the UNICEF-funded ECE program in Cambodia that reports cognitive improvements in 3-5 year olds, an impact evaluation specifically on the Bank-funded ECE programs indicates that while younger children made modest gains, 5-year-olds did worse.
The internal rates of return for all of the evaluated components are above the presumed opportunity cost of capital (10%). This indicates substantial levels of efficiency, and reasonable confidence around those estimates, for these components. The analyzed investments seem to have been cost effective and well spent, as the ECE and CCT programs represent the class of interventions with the best scholarly work demonstrating the highest-known impacts. However, because the evaluated components constitute only 10% of the total project cost, and in considering the challenges noted above, the efficiency of the entire project is rated Modest..
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated