Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Primary Education Development


  
1. Project Data:   
ES Date Posted:
05/08/2000   
PROJ ID:
P006101
Appraisal
Actual
Project Name:
Primary Education Development
Project Costs(US $M)
 12.6  0.0
Country:
Belize
Loan/Credit (US $M)
 7.1  6.5
Sector, Major Sect.:
Primary Education,
Education
Cofinancing (US $M)
 1.26  n/a see below
L/C Number:
L3422      
   
Board Approval (FY)
  92
Partners involved
ODA/DfiD 
Closing Date
06/30/1999 12/22/1999
         
Prepared by: Reviewed by: Group Manager: Group:  
Linda Ankrah Dove
Alain A. Barbu Gregory K. Ingram OEDST

2. Project Objectives and Components:

a. Objectives
Improve the quality of instructional inputs into primary schools and improve student achievement levels by:

  1. improving the quality and relevance of primary teacher training .
  2. improving the quality, availability and efficient use of educational facilities and resources for teaching, learning and assessment.
  3. strengthening planning and management to enable the government to develop policies and implement program for greater cost-effectiveness of educational expenditures.

b. Components
1.a Introduce a new primary teacher training program. 1.b Expand teacher certification.
1.c Provide incentives for trained teachers.
1.d Train school principals in instructional leadership.
1.e Improve planning and delivery of teacher training.
1.f Strengthen Belize Teacher Training College through technical assistance and equipment.

2.1 Enhance school facilities, repair and supply equipment and furniture (50 schools)
2.2 Reduce overcrowding through piloting multiple shifts.
2.3 Develop a new curriculum
2.4 Provide textbooks and teaching materials.
2.5 Reform examination and assessment system

3.1 Establish/Strengthen Educational Planning Unit in Ministry of Education (MOE).
3.2 Develop a Management Information System (MIS); school mapping, staffing policy and school construction plan.
3.3 Renovate and equip three District Education Centers (DEC) for decentralization of educational management

c. Comments on Project Cost, Financing and Dates
Actual total project costs are not available in ICR. Annex 2 does not include costs and the title is misleading. Par. 69 provides estimated cost for civil works of $2.8 m., with loan disbursements of $3.33 m. after reallocations in 1998. Region explained that DfiD 's grants were not cofinancing and were in part complementary and in part incremental to the project, and that the qualitative and institutional components were not tightly knit into the design. Thus, the grant reduced the need for Bank financing which was reallocated to expand civil works and support project management. DfID's expenditures, however, were not available to the Bank ICR team and the borrower contributions are not clearly presented.


3. Achievement of Relevant Objectives:

Objectives were satisfactorily achieved, given that the project experienced two changes of government, discontinuity in Bank supervision staffing and in inexperienced management. The ICR finds that progress towards development objectives proceeded satisfactorily after the midterm review.
  • Based on the new school map, in excess of 131 schools were built and 46 were renovated. Six instead of three DECs were built so that over 52% of the loan was expended on civil works and over 25% on goods. The government absorbed the entire recurrent cost of the latter instead of a share.
  • Loan funds freed up by additional DfiD grants were reallocated to DECs and additional classroom expansion (quantity unavailable).
  • Qualitative objectives were partially achieved. About 66 % of teachers were trained (original target 80%) and demonstrated higher levels of teaching skills. Curriculum implementation and textbook cost-sharing and delivery and use of assessment techniques to measure educational quality require persistence and effort to overcome shortcomings. Multi-shift teaching was not achieved and overcrowding persisted.
  • The MOE achieved a rationalization of its organizational structure including integration of the Planning Unit but has not yet decided on the responsibilities of the six DECs.
  • Cost-effectiveness objectives were partially achieved through reducing student repetition and the number of over-age children in primary education. Inservice teacher training was introduced with cost-savings (unquantified) over pre-service training but fee-based textbook provision needs persistent effort. The project raised teachers salaries and proposed regular increments as recruitment incentives. While salaries are the highest in the Region, recruitment remains problematic, though the ICR does not explore the economic, labor market and occupational status factors that may underlie this failure. Salary expenditures have risen and remain a very high 98% of the education budget.

4. Significant Outcomes/Impacts:

The ICR attributes the growth in net enrollments by 10,000, nearly 24% of the primary age-group, to the investment in school facilities.
  • The borrower's comments indicate that the project's focus on primary education weakened the focus on other important educational priorities, and thus did not maintain whole-hearted stakeholder support. When the new government assumed office in 1998, the project served to advance a sector-wide policy framework in line with the Bank's sector-wide strategy for educational development.
  • Initial development of a monitoring and evaluation system was encouraging, including school effectiveness indicators (though MOE ownership and institutional capacity still require much strengthening). The ICR recommends that the M&E data should guide preparation of the proposed Education Sector Improvement Project. In discussion, the Region emphasized the important development of the student assessment system.
  • A strong working partnership developed between the MOE and school managers, though the ICR missed an opportunity to identify mechanisms that proved highly effective (apart from local involvement through the full project cycle). The ICR concludes the partnership will ensure better delivery of school inputs for the future.
  • Though budgetary objectives were not explicit, the project strengthened budgetary planning for the subsector. Budget allocations and expenditures on primary education increased during the seven years. Annex 2 shows an increase in recurrent budget expenditures over projections of 26 percent (1998 dollar), with increases of 9% percent per pupil and 11% per teacher. Annex 1 states that the annual budget for primary education increased by 19% (67% appears to be a calculation error). The ICR states that the government maintained its commitment to provide counterpart funds despite a restrictive fiscal policy in 1993. However, it also states that the DfID grant offset the "reduction in counterpart funds." This raises the fungibility issue since Annex 3 shows that the share of primary expenditures declined (6 %) as a share of the total education budget. Education's share of the total capital budget is declining also. The Region explained that this was probably appropriate since physical infrastructure in education is sufficient.

  • 5. Significant Shortcomings (include non-compliance with safeguard features):

    The Region emphasizes that at completion the government included the project in its recurrent budget, and therefore that sustainability was likely. However, several factors--a deteriorating macro-economic environment, the steep rise in salary costs, the intra-sectoral budgetary imbalance, and the current focus of the Bank on other levels of education, lead OED to the view that sustainability will remain highly dependent on external assistance., continued commitment by successive governments and achievement of cost-efficiencies. Efficiency gains are not evident from the data, other than modest reductions in repetition and gross enrollment ratios. Annex 3 shows that pupil-teacher ratios increased by only 2 percentage points while teacher numbers increased by 14 points.
    The ICR does not provide an account of achievement under the cost-effectiveness objective. The rise in per pupil expenditures may not have been cost-effective since under 4% of project expenditures were targeted at sector capacity building through training and fellowships of sector managers and school personnel (compared with nearly 15% for consultants). Qualitative components, such as curriculum and instructional materials were rather ineffective until late in the project. The qualitative and institutional components supported by the DfID grant were mixed in their effectiveness but no costs are available.

    Local ownership of the large research study component funded by DfID was not achieved. The ICR concludes that the resources were wasted because findings were published under consultants' names and not integrated or translated into MOE policy and action but this is somewhat inconsistent with the ICR's finding on p.19 that the research studies had significant impact on policy without further explanation.

    The ICR states that the partnership between DfID, the Bank and the MOE achieved a satisfactory level. But it also suggests that, when DfID grant funds were substituted for the Bank loan, the decision to reallocate the Bank loan "savings" to building DECs and schools was problematic. This inconsistency is not explained, though the ICR states that the partnership has deepened during preparation of the follow-up project.

    6. Ratings:ICROED ReviewReason for Disagreement/Comments
    Outcome:
    SatisfactorySatisfactory
      Satisfactory for physical objectives and marginally satisfactory on long-term qualitative, institutional and cost-effectiveness objectives. Satisfactory overall for a first project.
    Institutional Dev.:
    NegligibleModest
      The Region states that staff wanted to assess ID as marginally satisfactory (partial) but the new rating system forced a choice of "unsatisfactory". The ICR text indicates that institutional development achievements were satisfactory and identifies several areas for improvement. OED agrees and rates ID as modest to acknowledge the achievements made in a first project.
    Sustainability:
    LikelyUncertain
      The macro-economic environment has deteriorated, exacerbating the imbalances in intrasectoral budget allocations and modest gains in cost-efficiencies. The financial factors put at risk the otherwise likely sustainability of the project.
    Bank Performance:
    SatisfactorySatisfactory
      OED rates the Bank's performance as satisfactory but only marginally so. Technical advice during supervision was satisfactory. But Bank monitoring during supervision and at completion was weak.

      The ICR (Annex 6) omits to rate Bank performance at appraisal though the text describes it as partially satisfactory (par. 86). OED's satisfactory rating for supervision takes into account the improvement in performance after the midterm review (ICR p.17) and the borrower's comments (par. 105).
    Borrower Perf.:
    SatisfactorySatisfactory
    Quality of ICR:
    Unsatisfactory

    7. Lessons of Broad Applicablity:

    The ICR's lessons are relevant to all first projects in any sector. They include the need for detailed implementation plans including costs and budgets, the separation of monitoring from outcome indicators, an agreed division of responsibilities between central and local partners and external donors, and the financial implications of organizational restructuring.
    From an OED learning perspective, the operational experience emphasizes how a modest investment project can be effective in empowering borrowers and implementing agencies to develop and manage more ambitious sector policies and programs. Several factors appear to be critical.
    • External financiers can demonstrate to policymakers the effectiveness of targeting investments and recurrent expenditures in a subsector to achieve intermediate objectives while shifting priorities from the short-term towards long-term sector policy.
    • Design and implementation activities need to be sufficiently modest to allow managers and stakeholders to achieve demonstrable results in a timely manner. The progress made helps create positive conditions for learning from trial and error, for building confidence and for establishing ownership among local stakeholders, beneficiaries and civil society.
    • External partners must not only provide sound advice but also ensure that local ownership of monitoring and evaluation is strong and that data and analysis are coordinated across the components that they support. Investments in M&E capacity building can then add value because consensus can be built on the basis of facts that validate decisionmakers' choice of policy, strategy and use of budgets.
    • The Bank must be prepared to spend the resources necessary to enable staff to prepare counterparts and ensure projects are ready for implementation; to maintain close contact with other financiers' plans; to maintain continuity during supervision; and to continue dialogue during implementation with all stakeholders and political groups while maintaining the trust of policymakers and managers.

    8. Audit Recommended?  Yes

              Why?  A first project with important sustainability, ID, M&E and partnership innovations in a high priority sector consistent with the Bank's Education Sector Strategy. However, the ICR ratings require validation.

    9. Comments on Quality of ICR:

    The analysis of factors affecting implementation and outcomes is insightful and the implementation lessons drawn are relevant more broadly. But the ICR has major weaknesses.
    • Internal consistency of evidence presented is weak and some evaluative judgments are not convincingly supported. This casts doubt on the validity of the ratings.
    • Total costs including the borrower's contribution are not presented.
    • The expenditure trends presented in Annex 3 require much more explanation. In themselves, they do not present clear evidence of likely financial sustainability (apart from continued high levels of external assistance).
    • The ICR could have done better in evaluating the Bank's decision during implementation to approve the doubling of the size of the DEC component. This investment appears not to have been adequately justified. The Region states that the appraisal report provided little guidance or detail on the component but that it evolved during implementation.
    • While useful and generally positive, borrower comments on the ICR are presented in Annex I. The inclusion of a summary of DfID's evaluation findings (listed in the files) would have added depth and breadth to the ICR.
    • The ICR's inconsistencies might have been remedied in part through tighter editing, proof-reading and corrections.
      ICR Annex 5 shows ID rating as satisfactory. The Region states that this was an error and it should read unsatisfactory.
      Annex 6 omits to rate Bank performance at appraisal, though the text describes Bank performance overall as satisfactory and partially satisfactory at appraisal (par. 86). The Region states that borrower performance on preparation should read as satisfactory, though par. 82--86 assess it as unsatisfactory.

    © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions