Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Education For All-fast Track Initiative


  
1. Project Data:   
ICR Review Date Posted:
02/21/2013   
Country:
Cameroon
PROJ ID:
P116437
Appraisal
Actual
Project Name:
Education For All-fast Track Initiative
Project Costs(US $M)
 24.80  24.56
L/C Number:
Loan/Credit (US $M)
 24.80  24.56
Sector Board:
Education
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  08/25/2010
 
 
Closing Date
12/31/2011 12/31/2011
Sector(s):
Primary education (100%)
Theme(s):
Education for all (100% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Erik A. Bloom
Judyth L. Twigg Navin Girishankar IEGPS2

2. Project Objectives and Components:

a. Objectives:
From the Project Appraisal Document (PAD, para. 16) and the Grant Agreement (Schedule I), the project's development objective was:

"...to reduce class sizes by recruiting and retaining more teachers in the public education system, as well as deploying them in an equitable manner."

The objective included targets to reduce the pupil-to-teacher ratio from 50 to 48 and to decrease randomness in teacher deployment (see discussion in Section 10 below) from 0.43 to 0.42, as a measure of equity.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project had one component.

1. Support the contract teacher program in primary education (Appraisal: US$24.8 million; Actual US$24.6 million). The sole component supported the payment of salaries of 37,200 primary school teachers under the government's Contract Teacher Program. The program was an important government effort to hire teachers in a flexible and cost-efficient manner that offered qualified teachers a contracted position. The teachers supported by the project were contracted by the government between 2007-10, and the project provided support for their salaries. This was done retroactively,

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project cost:
The final cost of the project was US$24.6 million, compared to an estimated US$24.8 million. At closing, approximately US$200,000 was cancelled. The project supported retroactive financing for teacher's salaries, up to 20 percent of the total amount.

Financing:
The project was financed by a Catalytic Grant from the Education for All-Fast Track Initiative (EFA-FTI, now known as the Global Program for Education). Parallel financing was provided in the amount of US$12.6 million from the French Development Agency (AFD) to support the salaries of additional contract teachers.

Borrower Contribution:
The government of Cameroon provided US$180.1 million in financing to pay the salaries of contract teachers, outside of the project, in a parallel fashion. The government also contributed through a parallel IDA-financed project, the Education Development Capacity Building Project (approved May 21, 2005 for US$18.2 million), to the project's financial management system. The size of these contributions is not quantified in the ICR

Dates:
The project became effective on January 12, 2011 and closed on December 31, 2011, as scheduled.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High. Improving human capital figures prominently in the Government's development strategy (including the 2010 Poverty Reduction Strategy), the 2006 Education Sector Strategy, and the FY2010-FY2013 World Bank Country Assistance Strategy (CAS). Specifically, the Education Sector Strategy calls for expanding the use of contract teachers to reduce the pupil-to-teacher ratio. Likewise, the CAS makes specific reference to the World Bank's role in supporting the Education Strategy, administering grants from EFA-FTI.

The project supported efforts to reduce the pupil-to-teacher ratio. As noted in the government's comments on the ICR (Annex 7), the government saw the project as an important input in its strategy to decrease the teacher deficit. The government specifically requested support from EFA-FTI to strengthen this aspect of their education strategy.

Research shows that reducing class size contributes to improved learning for children, especially when the pupil-to-teacher ratio is high. At the same time, maintaining a minimum level of quality is important even in a system that is focusing on meeting the Millennium Development Goals on education coverage. Without adequate quality, children will not stay in school, and the money spent on school is wasted. This supports the CAS in its objective to improve the quality of education.

b. Relevance of Design:
Modest. The design of the project was straightforward, as its sole activity was to finance teacher salaries during a two-year period. The project used an existing model that the government had developed to formalize the role of contract teachers and ensure that they had minimum training and received acceptable compensation. The project focused on a major challenge: the lack of teachers, particularly in poor areas. Addressing this problem would reduce the burden on communities and poor households that often had to pay teachers out of pocket. The project itself was designed to fill a known financing gap and to ensure that additional funds were available above and beyond what the government and AFD provided. An investment loan model was chosen largely because it allowed financial management control and oversight. However, the project in many ways acted like a budget support project, with 63 percent of the grant disbursed in one payment (the original design called for three discreet payments). A longer-term approach that emphasized policy development (for example, how to target the placement of contract teachers and how to incorporate the community model into the contract teacher model) could have attained a similar impact with more sustainable results. Given the relatively short time frame of EFA-FTI, a more programmatic approach or a mixture of grants and technical assistance may have been more appropriate.

For the project to achieve its development objectives, its activities would have to increase the total number of teachers in the public education system (assuming enrollment stays relatively stable) and ensure that these teachers are placed in an equitable fashion, in areas where the initial ratio is high. This approach would not function if the government used the project only to replace more costly civil servant teachers with contract teachers or if community teachers were converted into contract teachers, with no net increase in the number of teachers. The project also assumes substantial financing from the government on a parallel basis and that government policy works in the same direction as the project.

The long-term value added of the project's design is less clear. While the money provided was certainly important (as outlined in the PAD and the ICR), project design did not explicitly include any technical support or provide for project implementation in a transparent fashion. This created the risk of leaving the government without the technical resources to continue with the contract teacher model and to make adjustments in the model based on evidence. Likewise, it appears in the design phase that no other alternatives were considered.

The project's objective to increase the number of contract teachers was well defined and there was a clear relation between what the project was to do (support contract teachers) and the objectives (decrease pupil-to-teacher ratio). The project's objectives related to equity were less well-defined. From the context of the PAD, they focused on regional equality but but this is not well articulated in the objectives. Nor is it clear that there was a mechanism to ensure that poorer regions would get the highest priority.


4. Achievement of Objectives (Efficacy) :


Reduce class sizes: Substantial.

The student-to-teacher ratio dropped from 48:1 to 45:1 (3 points) during the project period (2009/10 to 2011/12). This exceeded the targeted improvement of 2 points. This is an important reduction considering the short duration of the project and variable government spending on education. Based on information from the beneficiary survey (Annex 5), prior to the project the ratio was dropping by around 0.5 points per year. As shown below, the project was able to increase this drop to around 1.5 percentage points per year.

The ratio is a function of both the number of teachers and the number of pupils in the education system. The project directly contributed to the number of teachers, through its financing of contract teachers. The ICR shows that the number of community teachers (by around 14 percent) and civil servant teachers (by around 5 percent) has declined. These were largely replaced with contract teachers financed by the project. The number of contract teachers increased by 13,600, and the project also was able to continue to support the 23,600 contract teachers who were already in service (ICR, page 15). Despite the decline in community and civil service teachers, the total number of teachers increased, leading to a reduction in the pupil to teacher ratio. This represented a net gain of around 8,300 teachers (Annex 2, table 2.2), which is around a quarter of the country's teacher deficit of 30,000 teachers (Annex 7).

This ICR also reports that enrollment increased by 4 percent during the project period. In the counterfactual case of no project, it seems unlikely that the government would have been able to hire as many contract teachers, and the student-to-teacher ratio would have increased. Thus the results are plausibly attributed to the project.

The PAD cites the importance of strengthening the contract teacher model to provide more flexibility in the teacher workforce. The project has contributed to this goal as contract teachers are gradually replacing civil servant teachers and community teachers. The contract teacher model offers more flexibility both for the government and for the teacher than the traditional civil servant model, as it allows the government greater flexibility in hring and transferring teachers. Likewise, while community teachers have been used successfully in several countries, they raise issues of equity since not all communities have the resources to finance teachers and not all households can equally provide support.

Deploying [teachers] in an equitable manner: Modest.

The project aimed to improve the placement of teachers in lesser-served areas. The main project indicator for placement of teachers in remote areas was "randomness," which is a quantitative measure of the degree to which teachers are placed where needed as opposed to deviation from equitable placement (see Section 10 for further discussion of this indicator). The ICR reports that randomness increased from 0.43 (2009) to 0.48 (2011), an increase of 0.05. The target was for a reduction of randomness by 0.01 to 0.42. Although the ICR provides a detailed explanation of the indicator, it is difficult to interpret.

The ICR also presents other measures of the equity of teacher placement and shows that contract teachers were increasingly placed in underserved areas (compared to previous practices) and that the ratio is declining in many of the most underserved regions. For example, 42 percent of teachers were placed in three of the most underserved regions in the north of the country. In the North and Extreme North regions, this led to a decline of the pupil-to-teacher ratio from 66:1 to 60:1 and 77:1 to 71:1, respectively. In the case of the Extreme North (the least served region in the country), this decrease in the ratio happened at the same time that enrollment increased substantially (from 529,000 to 625,000). As with the rest of the country, contract teachers took up a bigger proportion of the teacher force, gradually replacing community teachers (page 15 and Annex 3).

Although the evidence suggests that the project did benefit all regions of the country, it did not appear to have a particular focus on poorer regions as implied in the development objective, and there do not appear to have been specific human resource allocation mechanisms in place to contribute to achievement of the equity objective. The evidence suggests that the biggest beneficiaries of the project were the relatively well-off Littoral and South Regions, which saw reductions in the pupil-to-teacher ratio of 21 percent and 10 percent respectively. In contrast, the reduction in pupil-to-teacher ratio in the North and Extreme North was 9 percent each. Annex 2 of the ICR provides a detailed review of placement data by region.

5. Efficiency:

Substantial. The ICR argues that the project's simple and straightforward design was a relatively low cost approach to achieving its objectives, since the methodology already existed and transaction costs were low. The ICR further argues that this is the lowest-cost approach to reduce ratio (in the context of growing enrollment) given the institutional reality in the country. However, there may have been other approaches that could have provided similar outcomes, such as providing an incentive for the government to finance contract teachers, subsidies for communities to finance teachers, and better targeting of resources to areas that most needed additional teachers. The PAD does not provide insight into the technical alternatives considered. Given the lack of analysis of the design of the project, it is not clear if any of these alternatives would have produced a more efficient use of resources.

The delay in project implementation reduced the project's effectiveness. It was largely due to a delay in the specific audit (explained below in Sections 8 and 11), which was not required for effectiveness and was included as a dated covenant two months after effectiveness. However, this delay did not lead to the project closing late, and virtually the entire amount of the grant was disbursed as intended.

The project was administered by the government with the support of the project unit in an IDA-financed project (see Section 2d). Administrative expenses were directly financed by the IDA project and are not included in this project's ICR. Since the project was able to share pre-existing resources, it is likely that administrative costs were reasonable.

Both the PAD and the ICR provide economic analyses with an economic justification for the project along with estimates of the project's sustainability with government funds. Neither of these analyses include an estimate of the project's economic rate of return.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The relevance of the objective was rated high due to its well-defined response to a specific and well articulated problem: the low number of teachers, particularly in poor areas. The design was rated modest. Although the project had clear mechanisms to reduce the teacher deficit, its focus on equity was less clear. Achievement of the objective to reduce class size was rated substantial, while the equitable deployment objective was rated modest. While equity was not fully addressed, the project was successful in increasing the pupil-to-teacher ratio as planned. Finally, efficiency was also substantial, due to the simple design and the cooperation between this project and other projects in the country.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The contract teaching model is well established in Cameroon and is quite likely to continue after the project ends. Comments from the grantee (Annex 7) and from AFD (Annex 8) confirm that most of the contract teachers remain in place.

The underlying assumption behind the project is that the government will be able to replace funding from EFA-FTI in the future to continue financed the contract teachers supported by the project. This is supported by the economic analysis in the PAD. The ICR notes that there has been some delay in the government increasing its share (section 5.2) but confirms that most contract teachers are being financed after project closing. However, given the government's variable financing of education in the past, there does not appear to be any guarantee that financing will be available in the future.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
Although not explicitly mentioned in the PAD, the project was designed to have a clear impact on the education system, in particular how teachers are recruited and deployed. This is a major issue in the education sector that is not often directly addressed by projects.

The design did not appear to draw heavily on technical analysis and did not offer any alternatives considered. While it may be cost efficient to rely on the project implementation unit of another project (the IDA-financed education project), this is not transparent and may be risky should either project develop problems.

The PAD's discussion of risk appears to capture many of the issues that might develop during project implementation. The risk mitigation actions appear to have been reasonable.

    Despite the project's focus on equity in the objectives, the project did not seem to take into account the linguistic and ethnic diversity of the country, including the Pygmy population. There was no evidence of technical work to study how to best improve equity.
      A major piece of fiduciary work, the specific audit (often referred to as the "special audit" in the ICR), was included as a dated covenant. The project design did not include the risk that the specific audit would not be carried out in a timely fashion. The PAD considered the project ready to implement.

      A previous EFA-FTI grant (a development policy operation) supported the development of this model. This project was designed as an investment project in order to improve the supervision of contract teachers through the Bank's regular supervision model. Project design also took into account financing and support from AFD and the government, outside of the project, as well as support from the IDA-financed education project for management and financial management functions.

      Quality-at-Entry Rating: Moderately Unsatisfactory

      b. Quality of supervision:
      The Bank carried out regular supervision missions and maintained good dialogue with the government and development partners throughout the project. The Bank also worked to ensure that the activities of the project were integrated with other Bank activities in the country. The Bank responded proactively to problems in financial management. Central to this was the decision to delay disbursements until the specific audit was finalized and the results were released.

      The team did not focus sufficiently on the preparation of a reporting mechanism to monitor the hiring and reimbursement of contract teachers. This shortcoming led to delays in disbursement that had a negative impact on the quality of supervision.

      Quality of Supervision Rating: Moderately Satisfactory

      Overall Bank Performance Rating: Moderately Satisfactory

      9. Assessment of Borrower Performance:

      a. Government Performance:
      Overall, the government showed commitment to the project and its objectives. The government continued to support the contract teacher model and provided the bulk of the funds needed to operate the model (outside of the project).

      At the same time, the ICR notes that the government was slow to increase its financial support for contract teachers. It also created a complicated administrative structure that required approvals from the Prime Minister's Office, the Ministry of Finance, and the Ministry of the Public Function to operate (page 7). This could have been streamlined. The government could have better encouraged the timely completion of the specific audit.

      Government Performance Rating: Moderately Unsatisfactory

      b. Implementing Agency Performance:
      The implementing agency was the Ministry of Basic Education through the Monitoring Committee, with support from the IDA-financed education project. The Ministry carried out the necessary contracting of teachers throughout the country, including increasing the number of teachers. The implementing agency was slow to carry out the financial management covenant in the grant agreement and to introduce reforms and internal controls. The Ministry was also slow with a number of financial reports and data necessary to monitor the project; however, these were all eventually submitted.

      Implementing Agency Performance Rating: Moderately Unsatisfactory

      Overall Borrower Performance Rating: Moderately Unsatisfactory

      10. M&E Design, Implementation, & Utilization:

      a. M&E Design:
      The M&E framework was relatively simple, with two PDO indicators and one intermediate indicator. The first PDO indicator, ratio, matches closely the PDO and is relatively straightforward to measure and to use for policy purposes. The second PDO indicator, the randomness of contract teacher placement, is not clearly explained in the PAD, although the ICR does provide a useful explanation of its calculation (para. 62). It is a measure of the degree that the placement of contract teachers varies from an equitable distribution, where all teachers have the same number of students. Although it is commonly collected in African countries, it is difficult to use for policy purposes and the data from the project shows that the indicator varies significantly from year (Chart 2, Page 16), which suggests that it may simply be capturing random shocks. There are a number of alternative measures of equity of teacher placement (for example, the "distance" of a region from the national average or the variance in the number of contract teachers per region) that are included in the ICR and are more transparent.

      Even given the limited capacity in Cameroon to collect and analyze data, there was still room for additional indicators, including intermediate indicators. While the project itself was of short duration, a stronger M&E framework could have contributed to the development of the contract teacher model in the future. Some areas that may have been tracked include the retention rate for contract teachers, the proportion of community and civil servant teachers, the quality of contract teachers, among others. Many of these data are presented in Annex 2 of the ICR.

      b. M&E Implementation:
      The project monitored the indicators through the Monitoring Committee, although several problems were reported. According to the ICR, there were delays in establishing the education data system that was financed by the IDA project. There were also discrepancies among different data sources. In particular, the randomness indicator suffered due to a lack of data at the school level (page 16; data were available at the regional level.).

      a. M&E Utilization:
      Given the short duration of the project, the M&E system played little role in project decisions or policy development. At the same time, the amount of data presented throughout the ICR, particularly in the government's comments (Annex 7), suggest that more data are available than were formally used in the project. Annex 7 shows that the government is using data to focus on the teacher deficit.

      M&E Quality Rating: Modest

      11. Other Issues:

      a. Safeguards:
      The project was category "C" and did not trigger any safeguards during project preparation.

      b. Fiduciary Compliance:
      The project had a number of financial management issues. A weak financial management system was identified at entry, and a number of steps were introduced to strengthen internal controls. Despite this effort, the project was slow to implement a required specific audit, which consisted of both a financial audit and a technical audit. While the audit was generally positive, it did show that some internal controls were not in place and found evidence that a small number of teachers were misclassified. While most teachers appeared to be properly hired, there were delays in payment (Annex 8). At the same time, many reports (primarily the quarterly financial monitoring reports) were submitted late. It should be noted that the Bank did not disburse the first two payments of the grant until the audit was carried out, and it did not make the final payment until the results of the audit were released. The project did not have its own fiduciary staff and relied on the fiduciary staff of an IDA-financed education program.

      c. Unintended Impacts (positive or negative):
      The project contributed to the equity of education financing and to support efforts to eliminate the cost ("out of pocket costs") of primary education for households. The ICR documents that the project led to a reduction in community-supported teachers.

      The growth of the contract teacher model appears to have to contributed to the professionalization and quality of training of teachers (as discussed in Annex 6). This follows from requirements in the operational manual that teachers have a minimal level of quality. As a result, teacher training schools have raised their standards which has affected all teachers. This has led to some improvement in the training of community teachers and has contributed to creating a culture of evaluation in the education sector.

      d. Other:



      12. Ratings:

      ICR
      IEG Review
      Reason for Disagreement/Comments
      Outcome:
      Moderately Satisfactory
      Moderately Satisfactory
       
      Risk to Development Outcome:
      Significant
      Significant
       
      Bank Performance:
      Moderately Satisfactory
      Moderately Satisfactory
       
      Borrower Performance:
      Moderately Unsatisfactory
      Moderately Unsatisfactory
       
      Quality of ICR:
       
      Satisfactory
       
      NOTES:
      - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
      - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

      13. Lessons:
      The lessons presented in the ICR (page 23) are useful and should prove valuable to the government and to future IDA and EFA-FTI projects. In particular:
      • Developing systems requires informed risks. Although the contract model was already established, the project was designed to deepen the government's ownership. The Bank was able to take advantage of its knowledge and experience in Cameroon to work with the government.
      • Data are fundamental. While the project started out with a baseline, data collection was slow and did not appear to be used by the education sector.
      • Simplicity in design can facilitate project implementation. The project had a simple and easy-to-understand design. This allowed both the government and the Bank to diagnosis problems and resolve them. It also helped to support specific government policy goals.

      In addition, IEG offers several complementary lessons:
      • Simple projects can still benefit from complementary technical assistance and a robust monitoring and evaluation system to provide feedback to policy makers and the Bank. Technical assistance and research allow the government and the Bank to understand different options to improve the efficiency and equity of interventions.
      • Dated covenants do not substitute for project readiness. One of the main justifications for changing this project into an investment loan was the need to strengthen financial management in the project, and this was to be done through a special audit. Instead, this special audit was delayed until near the end of the project.

      14. Assessment Recommended?

      Yes
      Why?
      A more detailed assessment will provide guidance on two issues. First, the project is a good example of support by multi-donor partnerships (in this case, the EFA/FTI) and the role of the World Bank to provide value added. Second, the project is unusually simple in design, which may hold lessons for other projects.

      15. Comments on Quality of ICR:

      The ICR provides a frank and comprehensive assessment of the project. In particular, it provides detailed analysis that goes beyond the data included in the project's M&E framework. It makes a strong case about the project's impact, by comparing the country's current situation to several counter-factual situations. It also provides an honest assessment of some of the limitations in the design and implementation of the project.
      Minor shortcomings include the lack of information on the previous EFA-FTI project, and the parallel IDA project.

      The presentation of the randomness variable was not clear (it is also explained in the PAD) and the ICR could have made greater use of the data on equity in Annexes 2, 6, and 7.

      a. Quality of ICR Rating: Satisfactory

      (ICRR-Rev6INV-Jun-2011)
      © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions