Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Education Sector Project


  
1. Project Data:   
ICR Review Date Posted:
11/15/2012   
Country:
Ghana
PROJ ID:
P050620
Appraisal
Actual
Project Name:
Education Sector Project
Project Costs(US $M)
 78.0  74.14
L/C Number:
C3865
Loan/Credit (US $M)
 78.0  74.14
Sector Board:
Education
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  03/09/2004
 
 
Closing Date
10/31/2009 10/31/2011
Sector(s):
Primary education (41%), Tertiary education (37%), Central government administration (18%), Sub-national government administration (4%)
Theme(s):
Education for all (40% - P) HIV/AIDS (20% - S) Other accountability/anti-corruption (20% - S) Decentralization (20% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Susan Ann Caceres
Judyth L. Twigg Navin Girishankar IEGPS2

2. Project Objectives and Components:

a. Objectives:
According to the Project Appraisal Document (PAD p. 40), the objectives of the Project were to: (a) promote equitable access to, and efficient delivery of, quality services in pre-tertiary education; and (b) foster innovation, relevance, quality and efficiency in tertiary education. The same objective was reported in the Development Credit Agreement (p. 14).

The objectives were revised in a June 19, 2009 restructuring. The revised project development objectives were to: (i) improve equitable access to and completion of basic education in deprived districts; and (ii) improve quality of teaching and learning, management efficiency, and relevance of post-basic education (Project Paper, p. 4). The purpose of the revision was to make the first objective more precise and narrowly focused on basic education, and to focus the second objective more precisely on specific goals for tertiary education.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 06/19/2009

c. Components:
There were four components:

1. Sector Management Capacity Building (appraisal US$12.37 million; actual US$10.59 million) included the division of labor among organizations in the sector and within each organization; the leadership and direction in the sector and its organizations; general management procedures and practices; and management of key specific resources. This component had two sub-components:

  • Sector-Based Restructuring was to consist of operations that cut across and improved the working of the entire education sector, such as clarifying and improving the division of labor in the education sector as a whole by means of a horizontal and vertical functional analysis. It was expected that with decentralization new roles and functions would emerge for an optimal division of labor. This included an assessment of actual capacity, capacity gaps, and action plans to address deficiencies.
  • Agency-Based Capacity Building and Modernization consisted of operations to be carried out by individual organizations in the sector that aimed at improving capacity, as well as an assessment of actual capacity, capacity gaps, and the design of action plans to address deficiencies.

2. Pilot Programmatic Scheme (appraisal US$ 32.0 million; actual US$29.52 million) was to pilot a programmatic approach that the whole sector could adopt in the longer term. Basic education was to be targeted during the first phase of the project, and pre-tertiary education institutions thereafter. The scheme was to channel resources from the Ministry to district offices in support of non-salary recurrent expenditures. Funds were to be released on a quarterly basis against the performance of the sector in achieving the objectives of its previous year Program of Work (PoW). Indicators in the PoW were to be the basis for evaluating the Pilot Programmatic Scheme. The PoW focused on gender equity, decentralization, improved quality of education outcomes, increased participation of non-state actors, HIV/AIDS awareness, and early childhood development and care.

3. Tertiary Education Innovation (appraisal US$ 33.14 million; actual US$33.67 million) was to create a demand-driven Teaching and Learning Innovation Fund (TALIF) with the objectives of: raising the quality of teaching and learning, sharpening the relevance of skills and content in tertiary education, improving the efficiency with which institutions operated their academic programs, and opening up greater access in the disadvantaged Northern region for tertiary-level programs. Financing was to be provided through performance-based agreements to academic institutions.

4. Project Preparation Fund (PPF) Refinancing (appraisal US$ 0.49 million; actual US$0.36 million). No description is provided in the PAD (p.61).

After the June 19, 2009 restructuring, the following changes were made:
  • A sub-component to introduce a Skills Development Fund was added to the Sector Capacity Building component. The aim of the fund was to make skills training more relevant and better aligned with the requirements of the job market through a demand-driven and performance-based funding mechanism for Technical and Vocational Education and Training (TEVT) institutions.
  • Schools destroyed by the 2007 floods were to be reconstructed.
  • The focus of the Sector Capacity Building component was changed to improving the capacities of the District of Education offices in deprived districts, rather than the entire education sector.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: After a review of the entire IDA portfolio, the government reallocated resources to agriculture. Approximately US$7.0 million (SDR 4.45 million) was cancelled from the Pilot Programme Scheme component and reallocated to the IDA portfolio as part of a March 22, 2011 restructuring (ICR, p. 6). For this reason, actual project expenses were US$74.14 million, rather than the appraised value of US$78 million. The ICR's table of project costs by component (p. 26) shows a difference of only US$2.5 million between planned and actual costs for the pilot program component; it does not explain this discrepancy.

Financing: On March 22, 2011 the project was restructured at the request of the Government, to allocate resources to the IDA portfolio. This left total project financing to be US$ 74.14 million (SDR 48.25 million) (ICR, pp. 6, 26). US$ 45.17 million had been disbursed at the time the development objectives were revised.

Borrower Contribution: The Borrower did not provide a planned US$10 million in counterpart financing. The ICR (p. 6) reports that this did not have a negative impact on the components or project activities, since the project had adequate resources to implement the activities. The ICR does not explain why the Borrower did not provide the planned counterpart financing.

Dates: The closing date of the project was extended by two years from October 31, 2009 to October 31, 2011. This was granted to allow sufficient time to reconstruct schools (an activity that was added during restructuring), as well as complete the skills training pilots and have sufficient time to assess results. The project had a level one restructuring on June 19, 2009 to revise the project development objective, modify the results framework, reallocate funds between categories, drop the legal covenant on higher education fiance, and extend the closing date of the project (ICR, p. 5). On March 22, 2011 the project was restructured for the second time (level two) to reallocate resources from this project to agriculture projects (ICR, p. 5). On October 10, 2011, the project was restructured for the third and final time (level two). This was done to reallocate resources to the Pilot Programmatic Scheme to be able to complete all planned activities (ICR, p. 6).


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Relevance of Original Objectives: Substantial

The project's objectives were substantially relevant to the Bank's Country Assistance Strategy (2007-2011) and its Progress Report (2010), which has the strategic goal of improving equitable access to and participation and completion in quality education and training at all levels, as well as improving quality of teaching and learning, management efficiency, and relevance of post basic education. The project was directly aligned with the Government priorities outlined in its Education Strategic Plan (2003-2015), which included four main areas: equitable access to education, quality of education, educational management, and science, technology and technical and vocational education.

Relevance of Revised Objectives: Substantial
The project's revised objectives were also substantially relevant to the Bank's Country Assistance Strategy (2007-2011) and its Progress Report (2010), which support the focus on education services through decentralized systems, post-basic and innovative tertiary support, and technical and vocational education. The revised objectives were more consistent with the Government’s new National Education Reform Program and its emphasis on deprived districts and the need to provide resources into flood stricken areas. They were also aligned with the Government's focus on making skills training responsive to labor market needs.

b. Relevance of Design:
Relevance of Original Design: Modest

A Quality Enhancement Review in 2003 judged the design to be appropriate. However, the ICR and a Quality Assessment of the Lending Portfolio in 2008 found the design to be overly ambitious (ICR, p. 8). The original Results Framework was based on the Education Sector Plan, and as such, there was misalignment between objectives, components/activities, and outcomes. Nationwide rather than project-focused measures were created. Outcomes were not established for each objective. Measures with limited relevance to the project were included. (See Section 10 for M&E design weaknesses.)

Relevance of Revised Design: Substantial

The restructuring tried to resolve design problems, such as: focusing capacity building solely on district education administrators, rather than the entire system; restricting the focus of the project to deprived districts; supporting schools in Northern Ghana destroyed by the 2007 floods; and introduction of a skills development fund. These changes reduced the complexity of the project design. The restructuring focused on the project components where implementation progress was limited (i.e., Sector Capacity Building and Pilot Programming Scheme), rather than the component that had near fully dispersed (Teaching and Learning Innovation Fund). The Results Framework was revised during restructuring to better measure the impact of project activities.


4. Achievement of Objectives (Efficacy) :

Original Objectives

As noted by the ICR (p. 14), data are limited for the original objectives because the June 2007 ISR was the last reported information on the original indicators; moreover, data were not collected on every indicator. In addition, the Criterion Reference Test was eliminated. Criterion-referenced tests use an objective standard to provide information on what individuals can do, and not on how they score in relation to the scores of particular groups of persons. Between 2007-2009 another set of indicators was monitored, but these indicators were not approved by the Bank. Thus, limited data are available on the original indicators, and these indicators are based on nationwide data, rather data specifically linked to project activities. After restructuring, 25 original indicators were dropped.

Promote equitable access in pre-tertiary education: Modest. Without information on the project's activities relevant to this objective, it is not possible to establish a results chain linking the project's interventions to observed outcomes. Also, key targets were not achieved relative to this objective.

Outputs:

None provided in the ICR.

Outcomes:

Increase in national primary gross enrollment rate from 80% in 2003 to 95% in 2009. Target (98%) was partially achieved.

Improved gender parity as measured by increase in gender parity index at primary level from 0.92 in 2005 to 0.95 in 2011. Target (0.96) was partially achieved.

Increase in regional gross primary enrollment rate. The ICR reports that this indicator was dropped during the 2009 restructuring, as the Government wanted to target all deprived districts instead of just the three Northern regions (p. iv).

Region 2003 2007 Target

Northern 65% 76.2% 92%

Upper East 70% 84.4% 93%

Upper West 63% 81.5% 92%

Promote efficient delivery of quality services in pre-tertiary education: Modest

Outputs:

An educational management information system was used to generate annual reports in all 170 district education offices, meeting the target. The ICR reports that districts are now able to develop work plans that identify priorities, allocate appropriate resources, and implement activities. The ICR also indicates that the project supported the purchase of 282 computers, printers, and accessories, as well as Education Management Information System (EMIS) training (ICR, p. 28).

Studies were completed: review of internal division of labor in the Ministry of Education (MOE) and Ghana Education Service (GES) at the central, regional and district level; analysis of staffing needs and training at the MOE and GES headquarters, regional and district education offices; assessment of information needs and the development of a computerized human resource management system at the MOE, GES headquarters, and regional and district education offices; assessment of the physical facility and equipment needs for a fully decentralized education system and development of a computerized physical resource system at the MOE and GES; and review of accounting and reporting operations in the MOE and its agencies. The ICR reports that the Government used the results of these studies to develop and implement a job grading and job evaluation system called the Single Spine Salary Structure (ICR, p. 27).

Outcomes:

The ICR does not report information on the efficiency or quality of service delivery in pre-tertiary education.


Foster innovation, relevance, quality, and efficiency in tertiary education: Modest

Outputs:

The project built and improved learning facilities, revised and developed curricula, promoted staff development, and introduced competency-based training and problem-based learning. The ICR reports changes such as: incorporating information technology and communication in the curriculum, revising and updating polytechnic curriculum, and creating a series of internship programs that link institutions with employers (ICR p. 16). 75 new courses and 10 new programs were created and 109 courses were upgraded. 82% of new polytechnic and post graduate programs were assessed to be satisfactory. Target (75% of programs) was achieved.

Introduced a transparent and competitive funding mechanism for tertiary education. Target (mechanism established) was met.

Developed higher education policy based on principles of equity, efficiency, relevance, quality, and accountability. Target (policy) was met.

Outcomes:

The ICR states that supported programs under a Teaching and Learning Innovation Fund showed improvements in teaching, management, efficiency, or learning outcomes, with 64% of supported programs achieving their objectives, meeting the target of 60%. 82% of the 412 projects were completed and successfully implemented, but the ICR does not describe the criteria for measuring "successful implementation" (ICR p. 16).

The ICR does not provide direct information on outcomes related to equity, efficiency, relevance, or accountability.


Revised Objectives

Note: Outputs and outcomes noted above are not repeated under revised objectives, and also apply to the revised objectives.

Improve equitable access to and completion of basic education in deprived districts: Substantial

Outputs:

Re-constructed 67 schools in Northern region. Target (67 schools) was met. This included 458 boreholes and a number of water tanks and gender friendly toilets (ICR, p 31).

Used educational management information system to generate annual reports in all 170 district education offices. Target (100%) was met. The ICR reports that districts are now able to develop work plans which identify priorities and allocate appropriate resources and implement the activities.

Implemented annual programs of work in each of the 61 deprived districts to improve access and gender parity. Target (61 districts) was met. Training for 183 district education staff in procurement was provided by the project (ICR, p 31).

Outcomes:

Increase in primary gross enrollment rate in deprived districts from 76% in 2003 to 93% in 2011. Target (100%) was partially achieved.

Improved gender parity as measured by increase in gender parity index at primary level from 0.92 in 2005 to 0.95 in 2011. Target (0.96) was partially achieved.

Increase in primary school completion rate in deprived districts from 52.7% in 2003 to 87% in 2011. Target (89%) was partially achieved. The ICR did not discuss the extent to which government promotion policies contributed to the changes in the completion rate.

Increase in junior high school gross enrollment rate in deprived districts from 72% in 2009 to 77% in 2011. Target (86%) was not met.

Increase in the gender parity index in junior high school in deprived districts from 0.86 in 2009 to 0.93 in 2011. Target (0.95) was partially met.

Improvement in the gross primary admission rate in deprived districts from 82% in 2009 to 100% in 2011. Target (105%) was partially met. There was an 18.7% increase in this indicator (ICR p. vii).

The ICR notes that the indicators fell short of targets, but the trends show marked improvements during the life of the project. When comparing the achievements of project districts to the national average, the deprived districts had substantial increases in gross enrollment rates, gender parity, and completion rates (ICR p. 14-15). For example, there was a 16.9% increase in the primary gross enrollment rate in deprived districts, while the national average only increased by 9.9%. There was a 34.2% increase in the primary completion rate, while the national average only increased by 13.7%.

Improve quality of teaching and learning of post-basic education: Modest

Outputs:

Established nine distance learning programs. As noted by the ICR (p. 36), between 2006-2007 and 2009-2010, enrollment in distance education programs increased by 81%, thus, the project's investments in new and improved distance education programs were timely.

82% of new polytechnic and post-graduate programs were assessed to be satisfactory. Target (75% of programs) was achieved. The ICR does not describe the methodology used to assess the programs.

Participation in the Trends in Mathematics and Science Study (TIMSS). The ICR (p. 28-29) reports that as a result of this participation, Ghana has sector level disaggregated data and a benchmark for measuring learning outcomes. The project also supported work related to the third cycle of TIMSS (e.g. school sampling, dipatching test materials, data collection and test administration, preparation of scoring booklets, data entry and cleaning, training of trainers in all 61 deprived districts, participation in international quality control seminars and school coordinator and test administrator workshops).

Piloted and scaled up capitation grants in deprived districts. Capitation grants were implemented in all 170 districts. Target (170 districts) was met. The grants program provided non-salary resources at the school level to address the needs of the schools (ICR p. 15). The ICR reports that 52% of investments were used for school rehabilitation and furniture, 33% of investments were used to improve school sanitation, and 12% of investments were used to support needy pupils (ICR, p 31).

No outcome data are reported relevant to this objective.

Improve management efficiency and relevance of post-basic education: Substantial

Outputs:

Developed a Skills Development Fund, which has also been scaled up under a follow-up Bank-supported project. The ICR notes that an operational manual was developed to guide the funding process and 484 concept notes were submitted, of which 183 were invited to submit full proposals (ICR, p. 17). The Government has fully adopted the Skills Development Fund through a follow-up project (Ghana Skills and Technology Development Project, US$70 million, 2011-2016) as a result of the success of this activity.

Supported the development and establishment of the Council for Technical and Vocational Education Training (COTVET), which is responsible for the delivery of technical and vocational education.

Commissioned seven studies to guide the design of the Teaching and Learning Innovation Fund, including: Labor Market for Tertiary Graduates in Ghana; Expenditure and Revenue Analysis of Tertiary Institutions; A Vision of Postgraduate Education in Ghana; A Strategic Plan for Information and Communication Technology Development; Priorities and Strategies for Capacity Building in Tertiary Distance Education; HIV/AIDS in Tertiary Institutions; and Socio-Economic Background of Tertiary Students. The ICR reports that these studies guided the choice of interventions and activities under the Fund and helped link the proposed activities to the needs of the sector (ICR p. 33).

Established 16 HIV/AIDS prevention programs and 6 tertiary institutions now contain a workplace policy on HIV/AIDS.

Supported AIDS awareness campaign that reached over 2,000 students across polytechnic institutes and public universities.

Introduced a transparent and competitive funding mechanism for tertiary education. Target (mechanism established) was met. The ICR reports that 922 proposals were received during the five funding cycles, of which 412 (45 percent) were approved, supporting 27 public and private tertiary institutions and organizations across all ten regions of Ghana (ICR p. 35).

Developed higher education policy based on principles of equity, efficiency, relevance, quality, and accountability. Target (policy) met.

Construction and rehabilitation of 11 District Education Offices, furnished and equipped based on MOE standards. (ICR, p. 28).

Capacity building training for District Education Office (DEO) staff in the areas of financial management, data analysis, and fraud prevention. Specifically, 366 DEO staff were trained in fraud prevention, 487 DEO staff were trained in financial analysis and management, 40 staff were trained in procurement management, and 5 PIU staff were trained on SUN accounting software (ICR, p. 28).


5. Efficiency:


The PAD did not calculate an economic rate of return; however, it made an cost-effectiveness estimate for each component in the project (PAD pp. 26-27). Estimates for the Sector Capacity Building Component gains ranged from US$40 million by 2005 to US$ 80 million as of 2010, as result of the reduction of leakages and of administrative costs and gains related to reduction in absenteeism, higher staff productivity and better human resource management. The PAD noted that the other two components do not lend themselves to cost-effectiveness analysis, since the final outputs are not known. Yet, the PAD provided estimates for these two components, based on optimistic assumptions (e.g. boost the primary gross enrollment date in particular regions by 25 points over a 10-year period; 10% of students enrolled in institutions which were to benefit from TALIF grants and so would receive a supplementary allocation of about US$ 1,050 per student).

The ICR did not repeat the analysis conducted in the PAD, which was appropriate, since the assumptions used in the PAD were not applicable to the restructured project, and as previously noted appeared questionable in any case.

Cost comparisons provided in the ICR (p. 18) show that the average project cost of reconstructing a primary school with six classrooms (US$99,909) was slightly higher than DfID's cost estimate (US$96,000), while the unit cost of borehole construction under the project was slightly lower (US$6,250) than unit costs for NGO-supported projects (US$6,500). The average cost for the project to reconstruct kindergarten blocks was GH 83, 552, while USAID's estimate was GH 152, 000. The project effectively targeted Ghana's three poorest regions by providing greater levels of financial support in absolute terms and on a per student basis to the more economically disadvantaged regions (ICR p. 18).

The ICR did not describe the efficiency or value for money in the activities related to sector and agency capacity building and the Teaching and Learning Innovation Fund.

Efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Original objectives: Based on Substantial relevance of objectives, Modest relevance of design, Modest efficiency, and Modest achievement of three objectives, the outcome based on the original project objectives is rated Moderately Unsatisfactory (corresponding to a rating of 3 on the 6-point scale).

Revised objectives: Based on Substantial relevance of objectives, Substantial relevance of design, Modest efficiency, Substantial achievement of two objectives and Modest achievement of one objective, the outcome based on the revised project objectives is rated Moderately Satisfactory (corresponding to a rating of 4 on the 6-point scale).

The overall project outcome, based on the average of these ratings weighted by the amount of the total loan disbursed before (62%) and after (38%) restructuring, is as follows:

Outcome- original objectives = Moderately Unsatisfactory (3) x .62 = 1.86
Outcome- revised objectives = Moderately Satisfactory (4) x .38 = 1.52

Total: 3.38, or Moderately Unsatisfactory

The project's initial design was poor, and implementation prior to restructuring was slow and uneven. There was inadequate tracking of indicators, with limited evidence of attainment of several development objectives.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

While some of the project activities have become institutionalized (e.g. use of EMIS in all districts) and other activities (e.g. Skills Development Fund and APoW) have (or are expected to be) adopted by follow-up projects, there are significant risks to the development outcome. The fiscal situation of the Government was impacted by the 2009 economic crisis, and so resource constraints remain an issue. Quality of education is still a significant concern. Management capacity at all levels remains low, and there continue to be inefficiencies in the education system, making it difficult to improve education quality.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:

The Bank team was composed of technical experts, and preparation was spread over several months for dialogue and consultation to establish commitment to the project and its design. A Quality Enhancement Review was conducted, and its recommendations related to capacity building, decentralization activities, and competitive funding mechanisms were incorporated into the final design. The Bank identified specific institutional capacity constraints during project preparation, as well as risk mitigation measures. Implementation responsibility for each component was divided among various sub-sector departments.
The ICR reports that preparation was collaborative and technically sound, and based on lessons learned from previous operations (ICR, p. 6). Specifically, this project avoided the risk of developing a full sector-wide operation and instead focused on: sector capacity building, targeted support for basic education through the Pilot Program Scheme, and competitive funding for tertiary education (ICR, p. 6).

The Quality Enhancement Review judged the design to be appropriate. However, the ICR and a Quality Assessment of the Lending Portfolio found the design to be overly ambitious. The ICR indicates that the subsequent implementation problems related to the government's limited capacity, which was misjudged at entry. In addition, the Quality Enhancement Review was overly optimistic on the government's ownership and its commitment to decentralization and capacity building, which were necessary conditions for project success and key aspects of the project design. There were many weaknesses in the Results Framework (see Section 10). All of these shortcomings led to implementation delays and monitoring problems (ICR, p.21).

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:

The Bank maintained consistency in its supervisory staff, which provided continuity that was particularly beneficial given the high turnover among the Borrower's implementation staff. The project was restructured to bring more focus to the objectives and address design weaknesses. The ICR (p. 9) points out that this restructuring process took one and a half years to complete, which is unusually long and negatively impacted implementation, since design problems were not resolved more quickly. Many of the weaknesses in the Results Framework were resolved, but the new framework lacked robust measures for all the objectives and components. The Bank continued to provide moderately satisfactory ratings, even while project implementation progress was slow. Revised indicators were used for monitoring purposes in ISRs even though the project had not been formally restructured (ICR, p. 10). The Borrower's comments indicate that the Bank's country leadership instigated the cancellation of project funds when the rate of disbursement for the affected component (Pilot Programmatic Scheme) had increased, and so this was not a prudent decision (ICR, p 54).

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

Project effectiveness occurred three months after Board approval. The government was committed to improving the education sector and the situation in deprived districts. However, the Government did not have full ownership of the design, since plans to move forward with transferring power from Ghana Education Service officials to elected district assemblies were not implemented when the previous champions of decentralization disappeared within MOE. The Government instead transferred authority from Ghana Education Service headquarters to Regional and District Education Offices. The Minister of Education changed four times over the course of the project, which resulted in changes in policy direction and staff and impacted project implementation (ICR, p. 8). These factors contributed to the need to restructure the project.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:

There were three implementing agencies. Ghana Education Service had responsibility for the Pilot Programme Scheme. The Ministry of Education oversaw the Sector Capacity Building component. The National Council for Tertiary Education managed the Teaching and Learning Innovation Fund component.

The initial launch of the project included a workshop with representatives from the Ministry of Education, universities, polytechnics, directors from deprived districts, and representatives from the Ghana Education Service and the National Council for Tertiary Education. During the early implementation period, the Ministry of Education conducted five key studies that informed sector reform and the plan for District Education Offices.

Implementation was inconsistent across the life of the project, but after restructuring there was considerable implementation progress with 100% of the net credit disbursing and full completion of project activities. Frequent turnover of project implementation staff negatively impacted project performance. Insufficient district monitoring was provided by Ghana Education Service, and as a result districts were non-compliant in delivering financial management statements and procurement plans, which resulted in stopping the disbursement of funds in 2008 and 2009, virtually halting the implementation of one of the project components.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The Results Framework was based on the Education Sector Plan; as such, there was misalignment between objectives, components/activities, and outcomes. Nationwide rather than project-focused measures were created. Outcomes were not established for each objective, and some indicators were poorly defined (e.g. improved deployment and management of key resources, organizational structures of Ministry modified, etc.). There were too many indicators, and these often did not include baselines or targets. When targets were established, they were overly ambitious and unrealistic. Thus, the original Results Framework had the following design weaknesses: (1) indicators did not adequately measure project interventions; and (2) indicators were not attributed or linked to project investments. The Results Framework did not include a systematic accounting mechanism of the APoWs, and a framework for monitoring the Teaching and Learning Innovation Fund was not developed (ICR, p. 11).

The new Results Framework following restructuring was more closely aligned with project activities and improved the project's ability to measure progress toward achievement of the development objectives. However, there were still some weaknesses, since the indicators did not adequately measure all of the objectives (e.g. improve quality of teaching and learning, management efficiency, and relevance of post-basic education).

The project included an updating of the Education Management Information System (EMIS) developed under the Basic Education Sector Improvement Project (1996-2002) and expanded coverage to all districts. The ICR (p. 10) reports that the design for the EMIS activities was appropriate for the project.

b. M&E Implementation:

Data were collected, aggregated, analyzed and reported by each of the three implementing agencies. However, there was uneven implementation of monitoring and evaluation activities across the agencies. All district education offices used the EMIS system to collect and compile data for the Ministry. The National Council for Tertiary Education did not develop a monitoring and evaluation system to track the impact of the grants at the institutional level, despite repeated requests from the Bank. A formal assessment of the grant scheme was initiated in September 2011. The ICR (p. 11) reports that some of the Bank's indicators differed from those in the EMIS.

a. M&E Utilization:

The ministry used data collected from EMIS to generate the education statistical yearbook and Education Sector Performance Report (ICR, p. 11).

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The project was classified environmental Category B (O.P 4.01) for construction activities. The PAD reported that a Resettlement Policy Framework and Environment and Social Management Framework were completed, reviewed, approved, and disclosed by the Government and Bank (PAD, p. 31). The ICR notes that the Environment and Social Management Frameworks were revised and re-disclosed, given the new focus on rehabilitating the 67 schools damaged by the 2007 floods (ICR, p. 12). The project team reports that the Funds and Procurement Management Unit monitored environmental and resettlement policy compliance, including the establishment of water facilities (bathrooms and boreholes). The Bank monitored safeguard compliance and found it to be satisfactory.

b. Fiduciary Compliance:

Each implementing agency (GES, NCTE, and MOE) had its own finance and accounting unit with its own special account. The ICR reports that these arrangements were adequate and that at the time of project closing, there were no outstanding financial management reports. One of the three agencies submitted quarterly financial reports in a timely manner, while the others usually submitted delayed reports. Agencies fully complied with submission of the annual audited accounts. There was one qualified opinion in December 2008, and all other reports were satisfactory (ICR, p. 12). The project team reports that the issues in the qualified audit were resolved. The ICR indicates that there were weaknesses in the implementing agencies' performance, since there were frequent changes in accounting personnel, delayed submission of reports, and delayed provision of counterpart funds (ICR p. 12). The ICR does not provide information on procurement performance.

c. Unintended Impacts (positive or negative):
None

d. Other:
None



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Borrower Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

The following lessons are derived from the ICR (pp. 23-25):
  • A project's results framework should be explicitly linked to project objectives and interventions. The original framework for this project lacked a logical link between project activities, outcomes, and measures, and so it hampered the ability to measure progress.
  • Pilot programs must establish sound monitoring and evaluation systems to measure impact, identify problems and take corrective measures. The lack of an appropriate monitoring system limited this project's ability to regularly assess programs and outcomes. At the end of the project, an evaluation was conducted and so the lateness of this information prevented it from being used and disseminated.
  • A demand-driven competitive funding mechanism is an effective instrument for allocating public funding aimed at fostering innovation, relevance and quality in tertiary education as long as the selection criteria are clear, transparent, and well publicized. The funding mechanism for this project was linked to strategic priorities defined by beneficiary institutions and benefited from successful implementation.
  • The efficient and timely implementation of project activities is contingent upon building and sustaining project implementation capacity. Ongoing capacity building is needed, which was not the case in this project. The sporadic training coupled with staff turnover at the district level did not build capacity, which resulted in delaying project implementation.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:


The ICR presents the evidence concisely and is of good quality. It provides a good description of implementation experience and shortcomings. It gives a candid assessment of Bank performance. The Borrower's comments are thorough and useful. The ICR does not describe procurement performance.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions