Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Py- Second.ed Imprv


  
1. Project Data:   
ES Date Posted:
08/15/2002   
PROJ ID:
P007923
Appraisal
Actual
Project Name:
Py- Second.ed Imprv
Project Costs(US $M)
 40.50  34.20
Country:
Paraguay
Loan/Credit (US $M)
 24.48  24.50
Sector, Major Sect.:
Secondary Education,
Education
Cofinancing (US $M)
   
L/C Number:
L3941; LP217      
   
Board Approval (FY)
  96
Partners involved
 
Closing Date
12/31/2001 12/31/2001
         
Prepared by: Reviewed by: Group Manager: Group:  
Gillian M. Perkins
Martha Ainsworth Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
Support government initiatives to improve the quality, efficiency and coverage of lower secondary education and to increase the Ministry of Education's institutional capacity for information-based decision making. Specifically:

(a) build the necessary information base on educational system conditions and performance, essential for the design and monitoring of sectoral policies and strategies;
(b) improve teaching and learning conditions through provision of essential inputs and improved physical conditions;
(c) gradually ease overcrowding and expand access at the secondary level, beginning with grades 7-9;
(d) support more dynamic and participatory school management at the local level.

b. Components
[1] Institutional strengthening (9% of estimated costs): For the education system as a whole, (i) develop an educational statistics system for use at the school, regional and central levels, including provision of equipment and training; (ii) (jointly with IADB) finance TA, training, equipment, implementation of sample tests, and dissemination of results, for design and implementation of a student assessment system; (iii) contract research and policy studies to facilitate information-based decision making and increase the capacity for impact evaluation.
[2] Provision of textbooks and curricular support (16%): Support implementation of the revised curriculum for grades 7-9 through provision of (i) textbooks, classroom libraries, and basic classroom and teaching materials, (ii) related training for staff of the MoE's 18 state pedagogical units, and (iii) training, TA and equipment to the MoE's Learning Resources Division and local learning resource centers.
[3] Training (6%): Provide training to (i) teachers, to introduce the new curriculum and materials and upgrade teaching skills in core subjects, (ii) school supervisors, principals and administrators in education reform issues, curriculum application and school management, and (iii) members of parent associations on the reform and on school performance monitoring.
[4] Infrastructure upgrading and expansion (59%): Finance construction, rehabilitation and furniture for lower secondary schools.
[5] Pilot program for school-based innovations (3%): Provide grants up to US$4,000 to each of some 200 institutions for school-level initiatives designed to meet students' learning needs in core subjects.

c. Comments on Project Cost, Financing and Dates
The project closed on schedule with all of the Bank loan and 65% of the government commitment disbursed. Communities' contributions, at $2 m., amounted to about half the level estimated at appraisal. Despite the underrun in local expenditure, principally in the civil works category, all project components were fully executed. Disbursements lagged in 1997, when the local budget constraint was most severe, and regained momentum when the Loan Agreement was amended to increase the Bank's share of financing of infrastructure expenditures.


3. Achievement of Relevant Objectives:

The project substantially met all its objectives and exceeded quantitative targets in many cases.

Institutional strengthening: An education statistics system was designed and implemented at the central and regional levels, with plans developed for gradual implementation at the school level. Further training will be required to make full use of the statistics for analysis and planning and to reduce delays in publication of annual data. The national student achievement testing system was designed, with high quality TA from regional experts; appropriate training and procurement was carried out, and the system was successfully introduced. 14 sector studies were contracted and most were of satisfactory or better quality, although full use was not made of the study results for analysis, dissemination and feedback.
Provision of textbooks and materials: The project supported provision of free textbooks to all students in grades 7-9, exceeding original targets by 92% to meet an unexpectedly high increase in enrolments. An evaluation found the quality of textbooks and other materials to be generally good, but recommended more attention to cultural relevance and consistency across grades. The project did not succeed as intended in defining mechanisms for sustainable delivery of textbooks and materials.
Training: Quantitative targets were exceeded for all categories of training. The quality and results of training for parents' associations also exceeded expectations. In contrast, qualitative performance of the teacher training subcomponent is considered only "passable." Strong efforts were made during implementation to redirect the training towards the more participatory, locally-delivered model envisaged in project design. By the end of the project, promising pilots had been launched for in-service training which may be expanded under the next Bank-funded project.
Infrastructure: Construction and furnishing of classrooms exceeded targets at substantially lower than estimated US$ unit costs: total expenditure under this component was only 63% of the appraisal budget. (Depreciation of the Guarani helped to make this possible.) 1,720 classrooms were built by the Ministry and 1,230 by parents' associations, creating more than the planned 46,000 additional student places. Equity objectives were partially achieved: the schools selected for upgrading were the poorest in their communities but some of the poorest states did not benefit. Effective support was provided for improving preventive maintenance of school infrastructure, although maintenance capacity is still a concern.
School-based innovations: This component was successful in funding 250 small, school-based projects (200 were planned) to support curriculum guidelines on teaching methods, benefiting about 36,000 students.


4. Significant Outcomes/Impacts:

The project established the country's first reliable system of education information gathering to support planning and policy development and contributed to building a culture of evaluation and analysis.
  • As measured by the new student assessment introduced under the project, mathematics scores in grade 9 increased by 10% from 1997-99, while language scores fell slightly (more recent data would provide a better indication of potential project impact).
  • The project contributed to improvements in promotion and repetition rates. The progression rate of students moving from grade 7 to grade 8 increased from 82% in 1995 to 94% in 1999, while the share of students repeating grade 7 declined from 2.4% in 1999 to 1.3% in 2001.
  • Local publishers improved the quality of their textbooks and other education products.
  • Enrolment rates for the relevant age group rose from about 42% during project preparation to 72% by 2000 (the appraisal target was 57%), and the rural-urban differential in enrolment rates was reduced. The impact on overcrowding is not reported.
  • The process and quality of school infrastructure maintenance was improved through parents' inputs and the development of a practical manual which is considered a model in the region.
  • The project demonstrated the potential benefits of a new participatory model of school management and helped to build momentum for decentralization. The costs of school construction carried out by parents' associations were estimated at two thirds those of private contractors and parental involvement also contributed to higher quality.
  • The Borrower's comments indicate strong commitment to consolidating and building on the advances made under the project.

5. Significant Shortcomings (include non-compliance with safeguard features):

The delivery of teacher training continued to be based on the traditional model of central, one-time delivery, rather than continuous training, designed with stakeholder participation, and delivered mainly on-the-job or locally. As a result, the anticipated gains in quality and cost effectiveness were not achieved under the project. In hindsight, the appraisal expectation for reform of the teacher training system was unrealistic, underestimating the resistance to change.
  • The project failed to define and introduce mechanisms for financing future delivery of textbooks and teaching/learning materials.
  • Weak linkages of curriculum reform, development of new materials and teacher training for the primary and lower secondary sectors were due in part to inherent difficulties in changing teachers' attitudes and teaching methods, but also to lack of coordination. Although coordination between the Bank and the IADB (which was supporting the primary sector) was effective in development of the education assessment and statistics systems, a stronger coordinating role was needed by government in support for curriculum reform.
  • 6. Ratings:ICROED ReviewReason for Disagreement/Comments
    Outcome:
    SatisfactorySatisfactory
    Institutional Dev.:
    SubstantialSubstantial
    Sustainability:
    LikelyLikelyThe strength of government commitment argues for a "likely" rating, although mechanisms have not been established for effective and sustainable delivery of textbooks and materials, or of in-service teacher training. Future benefits of the institutional strengthening component also depend on further support to help consolidate what has been achieved and institutionalize full use of the new systems.
    Bank Performance:
    SatisfactoryHighly SatisfactoryProject preparation was exemplary in the attention given to defining implementation arrangements and anticipating related training needs. Supervision was proactive.
    Borrower Perf.:
    SatisfactorySatisfactory
    Quality of ICR:
    Satisfactory

    7. Lessons of Broad Applicablity:

    This is an example of a comprehensive and complex project that had satisfactory outcomes and considerable impact despite major constraints, including frequent turnover of education ministers and an economic crisis. The project was preceded by strong policy commitment, and by highly detailed participatory preparation. Overall political support for the project, combined with broad-based ownership within the Ministry of Education, carried the project through periods of political and economic difficulty.
    • Flexibility in development and adjustment of components during execution allowed the project to maintain relevance in an evolving reform process.
    • Stakeholder participation at the school and local levels, promoted through several different channels under the project, led to improvements in the quality and efficiency of resource use.
    • Experience of the teacher training component demonstrated the obstacles to introducing new paradigms in entrenched education systems, and the benefits of an incremental approach, building support for change through piloting and dissemination.

    8. Audit Recommended?  Yes

              Why?  This is a comprehensive and complex basic education project in a middle income country that was largely successful in both outputs and outcomes. An assessment would help to document lessons for meeting EFA goals among the poor in middle income countries.

    9. Comments on Quality of ICR:

    The ICR provides a clear and comprehensive account of the project's strengths and weaknesses. The Borrower's Comments should have been translated into English to make them more accessible and give them due impact. Mission dates in Annex 4 appear inconsistent with dates in section 1.

    © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions