Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Higher Education


  
1. Project Data:   
ES Date Posted:
07/25/2002   
PROJ ID:
P039449
Appraisal
Actual
Project Name:
Higher Education
Project Costs(US $M)
 250  7.29
Country:
Hungary
Loan/Credit (US $M)
 109.39  1.53
Sector, Major Sect.:
Tertiary Education,
Education
Cofinancing (US $M)
   
L/C Number:
L4287; LP326      
   
Board Approval (FY)
  98
Partners involved
 
Closing Date
06/30/2004 10/23/2001
         
Prepared by: Reviewed by: Group Manager: Group:  
Helen Abadzi
Patrick G. Grasso Alain A. Barbu OEDST

2. Project Objectives and Components:

a. Objectives
The Higher Education Reform Project aimed at developing a system of higher education that: (a) responds to the nation's changed and changing social and economic needs; (b) operates efficiently; and (c) mobilizes greater private finance and creates a more equitable system for financing higher education.

b. Components
The project had six components: (a) policy and institutional development to support the introduction and strengthening of key policy and administrative reforms in allocation of student applicants and finance, administration of higher education, teaching programs, tuition charges, student loans, and private higher education ($7.4 million); (b) a higher education institutes (HEI) investment program to reduce costs by consolidating single-purpose institutions into multi-faculty colleges and universities ($201.5 million); (c) a management information system to finance development of strategy and process redesign, software development, upgrading of information technology networks, and training ($19.4 million); (d) a management capacity development to support increased management capacity in planning, financial management, information systems, academic reform, and policy analysis ($6.9 million); (e) a student loan program based on a state guarantee of repayment ($2.6 million); (f) a project management and preparation facility to finance the establishment and operation of the project ($5.6 million).

c. Comments on Project Cost, Financing and Dates
The loan was given in Deutsche marks (DM263.6 m) and was equivalent to US$109.39 million. The project became effective in June 1998 and was cancelled 3.5 years later, in October 2001. Only US$1.5 million of the loan was disbursed, mainly for goods and services. The reasons for the loan cancellation were given as the high cost for borrowing from the World Bank, lengthy and cumbersome procurement processes, and a preference for European Union funding, which Hungary expects to join in 2004.


3. Achievement of Relevant Objectives:

Some significant policy reform objectives were achieved or partially achieved. However, most of the project's components were not implemented or only partially implemented and all but $1.5 million of the loan was cancelled. It is very difficult to attribute the project's achievements to what was financed by the project. Achievements by objective and sub-objective include (pp. 6-7, ICR):

(1) Increase higher education's responsiveness: (a) Enrollments in higher education institutions among 18-22 year olds increased from 17% in 1995/96 to 30% in 2001, surpassing the target of 25%. However, it's not clear that this can be attributed to the project and, to the extent that construction is part of the reason, it was not financed by the Bank. (b) Enrollments and financial support to HEIs are now based on student demand, a major accomplishment. (c) Restructuring of educational objectives, structure, and content was only partially achieved; most degree programs have not been restructured. The ICR says that this may happen when Hungary joins the EU, but this can't easily be credited to the project. (d) The objective of developing more responsive governance, advisory and management functions was not accomplished.

(2) More efficient use of resources: (a) Ex-post financing of HEI deficits has been eliminated but "much of this change [in financing] occurred prior to Project Effectiveness" (p. 8, ICR). Needed information systems and management expertise have not been provided to ensure control over expenses. (b) The sub-objective of economizing on investment and development expenditures was only partly accomplished, with adoption of strategic planning by the HEIs and objective criteria for investment decisions, but with little transparency. (c) Administrative expenditures per student were reduced, however not due to improved management (as forseen in the project), but to higher enrollments. (d) Total spending per student decreased and the FTE/teacher ration increased to 12.5 (against a target of 10.5). However, this was due to increased student-teacher ratios, not to technological improvements in teaching (as forseen in the project). (e) The sub-objective of managing expenditures on student services more efficiently was not achieved.

(3) Mobilize private resources and improve equity of finance: (a) The evidence offered on the project's success in mobilizing additional private resources within public HEIs is incomplete. It could not be determined whether the sub-objective of increasing tuition revenue to at least 15% of instructional costs of public HEIs was met. The ICR maintains that private funding of some public HEIs has increased due to tuition charged to non-state-funded students and from contracts, and that enrollments have grown more rapidly than state-sponsored students. However, the government also abolished tuition for first degree students, in conflict with this sub-objective. (b) Raising the contribution of private higher education as a share of national higher education expenditures was not accomplished. (c) The equity objective was not accomplished: the project did not succeed in providing selective state support of students on the basis of financial need, and the decision to provide "full tuition subsidies for first degree students combined with non-selective living allowances to state-supported students result in high inequity" (p. 7, ICR).

To summarize, of the 12 sub-objectives, 4 were fully accomplished (although in 3 cases not necessarily due to the project), 4 were partially accomplished, and 4 were not met at all, including the equity sub-objective, a major rationale for the project.


4. Significant Outcomes/Impacts:

The project was highly successful in implementing some policy and institutional reforms in areas such as allocating students and financing to institutions on the basis of student demand, updating teaching programs and making them more flexible, and creating a student loan program. Enrollment of 18-22 year olds rose to 30%. While student enrollments increased by about 65%, between 1995 and 2000, public spending on higher education increased only by 16.4%.

5. Significant Shortcomings (include non-compliance with safeguard features):

The government that had participated in preparing the project lost an election one month after the project became effective. The new government abolished the recently imposed university tuition fees and asked that the loan conditions and policies be restructured. The Bank had not anticipated the election results and that the new government would not want the project as designed; there was no contingency plan for this event.
  • While a student loan program was set up, the design was modified after the ICR mission, contrary to advice from international consultants, requiring significant continuing GOH subsidies that may restrict future access of students. The consultant team of the student loan program resigned their positions.
  • Management capacity was not strengthened and the management information systems which would give valuable feedback on university operation did not become operational.
  • It is unclear whether improvements in financing were brought about by the project.
  • Only $1.5 million of the $109 million loan was disbursed after 3.5 years of project effectiveness, indicating that most of the anticipated components were either not implemented or implemented with non-Bank funds.
6. Ratings:ICROED ReviewReason for Disagreement/Comments
Outcome:
SatisfactoryModerately UnsatisfactoryThere was progress in reform of some policies and enrollments increased more than expected. However, many components were only partially achieved or not achieved at all, and it isn't clear to what extent the policy reforms that did occur can be attributed to the project. Only 1.4% of the loan was disbursed.
Institutional Dev.:
SubstantialModestBasing university financing on enrollments was a major institutional achievement; however, tuition remained free and most degree programs were not restructured. It is unclear whether financing allocation changes were a result of the project.
Sustainability:
LikelyNon-evaluableWhile government commitment to some of the policy objectives is high, commitment to others is limited. The ICR notes considerable uncertainty about future financing by the government for continued implementation of project activities, including the student loan program.
Bank Performance:
SatisfactoryUnsatisfactoryDepite prior sector work and sound technical advice in project design, the project was extremely complex, there was no contingency plan in the event of a change in government, and the demands of the civil works component were underestimated. Supervision missions provided policy advice that was appreciated by the borrower but were not successful in accelerating implementation of the components.
Borrower Perf.:
SatisfactoryUnsatisfactoryWhile some policy reforms were embraced, the borrower's performance in implementation was uneven. Important policies were reversed, Government changes in implementation arrangements had an adverse impact on implementation, PMU management was weak, and most of the components were only paritally implemented or not implemented at all.
Quality of ICR:
Satisfactory

7. Lessons of Broad Applicablity:

- Policy reforms are more likely to succeed if consultations are held with multiple stakeholders, including the opposition. It may also be useful to include political experts on preparation teams to provide guidance on minimizing the risk of policy reversal and to advise on how best to appeal to the interests of stakeholders and political actors who may potentially oppose reforms.
-When a project expects to significantly change behaviors and rules of the administration, institutional experts should be included on preparation teams to advise on strategies and priorities for doing so.

-For complex project designs, one must think through clearly priorities and the sequencing of actions and build in implementation flexibility. Some project components may advance quickly, and others may lag.

-When planning large scale civil works in a country where the Bank lacks much experience, it is useful to engage local experts to lay out all the steps, estimate the time involved with each, and advise on how to optimize implementation time.

-Having Bank procurement expertise in a regional Bank office facilitates project procurement but by no means guarantees success.

8. Audit Recommended?  Yes

          Why?  OED and the region disagree on the ratings. Policy changes are difficult to attribute to the project without more extensive investigation.

9. Comments on Quality of ICR:

The ICR is marginally satisfactory. It outlined clearly the events of the project. However, it could have presented more evidence that the successful policy changes were a result of the project; it does not consider the counterfactual.

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions