|1. Project Data:
ICR Review Date Posted:
|Education Sector Support Project 1
Project Costs(US $M)
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|General public administration sector (45%), Tertiary education (38%), Secondary education (13%), Health (2%), Primary education (2%)|
|Education for all (40% - P)
Nutrition and food security (20% - S)
Child health (20% - S)
Decentralization (20% - S)|
||ICR Review Coordinator:
|Susan Ann Caceres
||George T. K. Pitman
|2. Project Objectives and Components:|
a. Objectives:According to Development Grant Agreement (DGA, p17), the Project Development Objective (PDO) objective is
"to improve the education quality in the Recipient’s territory through development of a broad education sector policy, enhancing the capacity and number of qualified teachers, enhancing the capacity and quality of education service delivery, and improving learning outcome at all school levels."
The Project Appraisal Document's (PAD, p12) statement of the PDO is:
"The project will improve education quality by improving the conditions and processes of teaching and learning at the school level, and the capacity for education service delivery across the education system by: (i) increasing the number of qualified teachers, (ii) improving the capacity and the quality of education service delivery by fostering community participation and by strengthening the management of human and financial resources at district and primary school levels, and (iii) improving learning outcomes at all levels by providing enhanced conditions for school effectiveness that will contribute to changes in the behaviors of teachers and students."
While the statement in the DGA and PAD are worded differently, there is no substantive difference in the objective. This ICR Review uses the version in the DGA.
b. Were the project objectives/key associated outcome targets revised during implementation?
c. Components:1: Teacher capacity development (appraisal US $15.50 million; actual $16.24 million). The component would support efforts to improve quality and expand capacity of teacher development and training at all levels through: (a) refurbishment of education facilities in universities (Chancellor College, Malawi Polytechnic and Mzuzu University); (b) construction of a new teacher training college; (c) staff development for lecturers and teachers; and (d) student assessment surveys to establish a baseline against which student achievement could be measured.
2: Quality improvements and inputs (appraisal US $3.70 million; actual $3.90 million). This would improve the conditions of learning at selected secondary schools through: (a) refurbishment of four national secondary schools; and (b) provision of additional textbooks for graduating teachers deployed to the Community Day Secondary Schools (CDSS).
3: Mitigating externalities affecting the quality of education (appraisal US $3.00 million; actual $3.22 million). This would enable provision of a school health and nutrition package to all primary schools through the distribution of Vitamin A and iron-folic acid to school children under 10 years old, de-worming, treatment of malaria and fever, and the promotion of good health and nutrition practices.
4: Direct support to primary schools (DSS) (appraisal US $3.70 million; actual $3.90 million). This component would enable supply of basic learning materials to schools while strengthening the participation of communities in school management through: (a) provision of grants to all public schools and District Education Management (DEM) units; (b) media campaigns; and (c) training of education sector staff at all administrative levels.
5: Capacity building and policy development (appraisal US$ 1.40 million; actual $1.54 million). This would be done through: (a) national education policy consolidation and capacity building as evidenced by (i) development of a medium to long-term education sector strategic plan; (ii) development of a teacher education policy, strategy and model; (iii) initiation of discussions on the formulation of a local language of instruction policy; (iv) supporting implementation of the new primary curriculum; (v) building capacity and develop fiduciary procedures for a wider SWAp; (vi) preparing operational plans for outsourcing hostels and catering services for higher education institutions; and (vii) training secondary school managers; (b) support to the implementation of Government decentralization policy through; (i) development of a training and capacity building plan for central and decentralized staff; (ii) supporting implementation of training activities at district level; and (iii) financing other inputs for institutional strengthening of the districts.
6: Project Implementation Unit and Contingencies (appraisal US$ 4.90 million; actual $3.33 million). The component would support the physical implementation and the management of fiduciary and procurement issues. (Education Development Management Unit- EDMU)
d. Comments on Project Cost, Financing, Borrower Contribution, and DatesProject Cost: Malawi experienced high inflation between 2004,when the project was prepared, and 2007-08 when the project began the civil works components. As a result, construction cost exceeded the original costs estimates and created a financial gap in the project (ICR, p. 7-8). By mid-term review, the total estimated cost overruns were at US$4.5 million. Some of this gap was mitigated by net SDR exchange gains of US$1.7 million, but an estimated US $2.8 million deficit still remained. As a result, there was a reduction in the overall civil works component of the project and reduced procurement of equipment for some facilities. For example, civil works in the national secondary schools was limited to refurbishment and planned construction in the university colleges was reduced, as well as the procurement of equipment for the facilities.
Financing: The IDA Grant was US $22.00 million at appraisal. By project closing US $21.87 million had been disbursed and US$0.13 million was cancelled.
Borrower Contribution: No borrower contribution was planned or made.
Dates: The project was extended by three months from its original closing date of September 15, 2010 to December 15, 2010 in order to finalize procurement of some items for civil works.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:Substantial. The 2002 Poverty Reduction Strategy Paper (PRSP) identified the enhancement of human development as one of Malawi's formidable challenges with a goal of reducing primary dropout and increasing secondary enrollment. The Malawi Government's Growth and Development Strategy (2006-2011) further prioritized improvements in education access, quality and equity across education levels in a bid to foster human development for growth. The objective is relevant to the country's conditions and strategies in the education sector Policy Investment Framework (PIF) 2000-2012, the Malawi Education for All paper, and the National Strategy for Teacher Education and Development (NSTED) (PAD, p. 7). For basic education, the focus in these documents was on providing a sufficient supply of qualified teachers and essential teaching and learning resources to achieve improved learning outcomes and reduce dropout and repetition rates. At post-primary levels, the emphasis of these documents was to produce more graduates in order to provide the number of teachers required at other levels of the education system and demands for qualified labor from other sectors of the economy.
The Malawi 2004-2006 Country Assistance Strategy (CAS) identified the strengthening of service delivery and safety nets as one of the three strategic pillars. In addition, investing in people was one of the pillars that underpinned the World Bank 2004 Strategic Framework for Assistance to Africa with an aim of supporting achievement of the education Millennium Development Goals. The Bank's 2005 Education Sector Strategic Update stressed the importance of key inputs to education service delivery including teachers, and quality learning environment. The subsequent Country Assistance Strategy (2007-2010) prioritizes its involvement with economic growth and so focuses its support to education in relation to vocational training and skills development. However, the project continued to be substantially relevant with its focus upon improving basic education, given that Malawi is unlikely to achieve universal access to primary education (CAS 2007 p.42).
b. Relevance of Design:Modest. The project activities and components are related to the objective. The project design emphasizes improving education quality by supporting the capacity of the Ministry via sector policy work, as well as it emphasizes increasing the supply of teachers and building civil works projects to increase the number of pre-service primary and secondary education teachers the system can train. Project design has some emphasis upon improving the conditions for learning through school resources, health packages, school grants. However, the components (outputs and outcomes) in the Results Framework were not well aligned with the objective of improving quality and learning outcomes at all school levels. Outcome indicators related to teacher-pupil ratio, rather than measures of quality or learning outcomes. Improvements in the quality of secondary teaching were measured by the increase in the number of spaces at tertiary level (PAD p 34).
|4. Achievement of Objectives (Efficacy) :|
The Bank was only one of 12 donors supporting Malawi’s education sector, and the primary education sector in particular (PAD, p 16). Their interventions supplemented the Bank’s inputs and included teacher education; provision of learning materials and textbooks; classroom-based assessment; curriculum development, district capacity building; primary school building; and national training of staff. Many of these parallel activities will have affected the outcomes of this project but the extent of the spill-over impact on the Bank-financed activities cannot be determined.
Improve the education quality at all school levels: Substantial
- Provided Direct School Support (DSS) Grants to all 5,200 public primary schools in 2007/08 and 2008/2009 FY (original target was 2,000 schools).
- Increased the number of spaces in tertiary level training institutions for secondary teachers by 1,000 (target was 250).
- Increased the number of spaces in teacher training colleges of primary teachers by 605 (target was 500). This was achieved through the construction of two new teacher training colleges at Machinga and Liwonde (ICR p. 27).
- Refurbished and expanded Faculty of Education Malawi Polytechnic, Chancellor College, and Mzuzu University (incomplete- implementation gaps) (ICR p. 26-30).
- With the new Teacher Training College built by the project, the number of new qualified teachers per year will increase from 443 to about 1048, if the new teacher training model (1+1 which means that teachers receive one year of pre-service training and one year of monitoring while teaching) continues to be supported by the government.
- Supported the development of Malawi's first National Education Sector Plan 2008-2017 (NESP).
- Trained 29 lecturers with graduate training.
- Refurbished four national secondary schools (target 4). However, some works remain incomplete. In Blantyre only 6 of 8 hotels were fully completed; In Deza 6 hostels, and in Lilongwe 2 hostels; were not fitted with beds and lockers; and none of the textbooks and materials for libraries was supplied. Physical sciences and chemistry laboratories in Blantyre remain incomplete. (ICR p. 29-30).
- US$70,000 worth of textbooks and journals were provided for Mzuzu University.
- Provision of textbooks and guides for about 250 graduating secondary school teachers was not done because of shortage of funds (ICR p.30).
- Staff Development at the Primary Teacher Training Colleges; Distance Education; Pilot strategies for strengthening entry level capacity of student teachers; and teaching learning using locally available materials for primary teachers was not done because of financial constraint (ICR p. 28).
The supply of primary teachers increased by 9.8% between 2006 and 2010 (ICR p. 12) and this contributed to a reduction in the student-teacher ratio. From a baseline level of 100:1 in 2004, the student-teacher ratio improved to 88:1, meeting 60% of the target (80:1). The ICR (p 12) presents plausible explanations why the target could not reasonably have been attained. Specifically that the target was unrealistic; student enrollment had increased from 3.12 million to 3.24 million; wage incentives for teachers remained poor; and that the teacher attrition rate was 4.3% a year. However, the efforts of the project (civil works projects to create more training spaces and train lecturers to address the increased enrollment of preservice teachers) will most likely be realized in later years. The ICR presents no evidence about the effectiveness of the improved student-teacher ratio. In the region's reply to the draft ICR Review, updated data was presented which indicated that student-teacher ratio improved to 76:1 in 2010-11, meeting the target (80:1).
The student-teacher ratio at secondary level improved from 82:1 in 2004 to 70:1 in 2010, meeting 71% of the target (65:1). The ICR (p13) states the targets may have been missed because of unrealistic target-setting; excessive student enrollment compared with teacher recruitment; and/or high teacher attrition at the secondary level. The ICR presents no evidence about the effectiveness of the improved student-teacher ratio. In the region's reply to the draft ICR Review, updated data was presented which indicated that student-teacher ratio in the Community Driven Secondary Schools improved to 61:1 in 2010-11, meeting the target (65:1).
The Direct Support to School (DSS) grants fostered improvement of community participation and management of human and financial resources at district and school level. According to the ICR (p. 14) the innovative enrollment based grants to all the 5200 public primary schools enabled acquisition of non-textbook materials for teachers (an incentive for improved performance among teachers) and learners in the 2007/08 and 2008/09 FYs. In 2009/10 the partnership with DFID enabled the DSS to provide grants for minor maintenance. The DSS evaluation report indicated that: (i) 99% of the pupils in the sampled schools reported to have benefitted from this program; (ii) 70% of head teachers agreed that DSS reduced learner absenteeism and contributed to improvements in learner performance (89%); (iii) 95% of teachers reported that DSS led to the availability of teaching-learning materials in schools; (iv) DSS enabled active engagement of district assemblies with local community leadership and also resuscitated School Management Committees (SMCs) and Parent Teacher Associations (PTAs). The 2008 Public Expenditure Review (PER) reported that school inspection by District Education Management was increased to 71.4% (no baseline given),while all schools sampled were found to have active SMCs and PTAs that were attributed to the DSS.
Improve learning outcomes at all schools levels: Modest
- Baseline testing for the Primary Achievement Sample Survey was conducted in Math and English for Standards 3, 5, and 7 (ICR p. 15).
- School health and nutrition interventions were delivered to 98% of eligible and targeted primary school pupils (target was 80%). UNICEF and GTZ provided parallel financing for the school health and nutrition packages and DFID for the direct support to schools program. This was enabled by the implementation of the pilot national School Health and Nutritional Program (SHNP) which ran from 2007 to mid 2009 targeting children in all public primary schools. The program involved provision of the following drugs and vitamins to school children: Albendanzole for de-worming; Praziquantal for Bilhazia; Vitamin A supplements -Rentinol; Folic Acid and iron tablets; Paracetamol; and Fansidar.
The results from the baseline Primary Achievement Sample Survey was not provided in the ICR. The planned follow-up survey was not conducted due to lack of project resources (ICR p. 30). The 2008 Public Expenditure Tracking Study (PETS) also indicates that 84 percent of the schools received antimalarial and bilharzias drugs. The ICR states that no impact evaluation was undertaken to establish the effect of the program on learning outcomes.
In the region's reply to the draft ICR Review, additional data from the Ministry of Education's Primary School Certificate Examination was presented, showing some improvement between the beginning of the project (2006) and end of the project (2010), with a slight decline in 2007.
Table 1. Malawi School Certificate Examinations 2006-2011
Efficiency is rated Modest.
Economic and financial analysis was carried out in the PAD, comprising the analysis of: selection of a teacher training strategy, fiscal sustainability and cost-benefit computations. Several simulation models were run in order to determine the number of primary and secondary education teachers that would have to be trained so as to achieve the target levels of pupil to student ratios. With this basis, computations were done of the fiscal recurrent costs of additional teacher training, and of the increased salaries to be paid to more qualified teachers, under several assumptions of reducing Ministry of Education’s wastage and inefficiencies. In this manner, a scenario was arrived at whereby the expansion of the qualified teacher corps does not give rise to an increase in recurrent costs beyond the limit of 4 % of GDP in any given year. This exercise permitted computing the costs involved under different targets. An analysis of the costs and benefits of the project was conducted based, first, on the improvements in quality resulting from the reduction in pupil to teacher ratio, which would lead to reducing desertion rates and thus increasing the number of children who graduate from complete primary school. Given the increase in hourly wages associated with attaining complete primary education vis a vis the wages of those who did not finish primary school, and given the number of additional children having attained complete primary school, the benefits of the project were calculated. Other source of benefits originated in the iron supplementation package, given that those adults who do not experience anemia during childhood are known to obtain higher wages in the labor market. The computed benefit-cost ratio for the project was 2.71.
The analysis provided in the PAD was not updated at project completion, except to note the assumption used to calculate the teacher-student ratio and suggest that these assumptions may have been optimistic.
The ICR did not provide formal measures of efficiency, Instead evidence provided included:
- Reducing the number of years for primary school teachers preservice training from 2 to 1 was an efficient was to fast track qualifying teachers in to the primary education system, however, it is hard to know how much of the cost savings will be offset by the possibility of lowering teacher quality.
- An average unit cost of $28,000 was spent to train each lecturer in post graduate. International postgraduate training costs including living allowance are estimated to range from $17,000 to $35,000. The return of all 29 lecturer to their colleges upon completion of their post-graduate program enabled the colleges to expand their program offerings and increase student intake.
- The project enabled the Government to leverage additional resources from development partners through parallel financing partnerships generated during the course of project implementation.
Finally, the ICR notes (p 17-18): "The allocation of the limited resources for large infrastructure projects at the universities and colleges may not have been the most efficient use of resources. This is especially true in light of the reduced scope of the project....Despite these investments, there were still some construction projects that were not completed and all the equipment was not procured for the new facilities. This situation is compounded by the low quality of construction which is likely to impose unplanned maintenance costs to the beneficiary institutions."
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated