|1. Project Data:
ICR Review Date Posted:
|Jordan Employer Driven Skills Development Project
Project Costs(US $M)
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|Vocational training (69%), Central government administration (31%)|
|Education for the knowledge economy (50% - P)
Improving labor markets (50% - P)|
||ICR Review Coordinator:
||Judyth L. Twigg
||Lourdes N. Pagaran
|2. Project Objectives and Components:|
According to the Loan Agreement (2008), the original project development objective (PDO) was "to assist the Borrower in improving its technical and vocational education training (TVET) sector through the harmonization of policies pertaining to the development of said sector and the realignment of such policies with the operational functions of E-TVET, including the development of employer participation in the formulation of said policies and in carrying out institutional development reforms as well as in the design and delivery of a skills development program."
The Project Appraisal Document (2007) has a slightly different wording of project objectives, although the meaning is essentially the same:
To enhance the internal and external efficiency of the E-TVET sector by making it more flexible and demand driven through the development of employer community participation in (i) sector policy formulation, (ii) institutional development and reform, and (iii) skill development program design and delivery.
The wording of the Project Appraisal Document is used in this Review, as it is more outcome oriented than that in the Loan Agreement.
The objective was revised in March 2013. The revised objective is:
To realign the E-TVET sector with the National Employment Strategy by enhancing the enabling conditions for employer participation in: (i) TVET institutional development; and (ii) skills development program design and delivery by the Vocational Training Corporation (Restructuring Paper 2013).
During this Level-1 restructuring, four PDO indicators were dropped, one new PDO indicator was added, and the two remaining PDO indicators were revised.
b. Were the project objectives/key associated outcome targets revised during implementation?
If yes, did the Board approve the revised objectives/key associated outcome targets?
Date of Board Approval: 04/04/2013
The four components supported by the project are presented below, showing project cost estimated at appraisal, after the restructuring, and actual project costs. These amounts do not include the front-end fee for the IBRD loan, which is included in the total project costs reported above.
1. E-TVET System and Council Development with Employer Participation (Appraisal: US$ 1.90 million. Revised: US$ 0.90 million. Actual: US$ 0.73 million). This component was to support institutional development of the E-TVET Council and its Secretariat and implementation of key aspects of employment-related planning, development and training with TVET stakeholders through:
(i) E-TVET Council orientation and capacity development;
(ii) E-TVET sector policy, planning, and organizational development with active employers’ participation;
(iii) E-TVET system performance assessment with active employers’ participation; and
(iv) E-TVET sector promotion and awareness among TVET stakeholders.
2. Restructuring of Vocational Training Corporation (VTC) (Appraisal: US$ 6.14 million. Revised: US$ 6.14 million. Actual: US$ 2.93 million). This component was to increase completion rates in training courses and to provide incentives for improved job relevance and quality of training through:
(i) Establishment of the mandate, governance structure, and regulatory framework for the new Skills Development Agency;
3. Training and Employment Fund Development (Appraisal: US$ 0.06 million. Revised: US$ 0. Actual: US$ 0). This component aimed to make the Fund more responsive to labor-market skill needs. It was to finance technical advisory services and related activities to support the establishment of a demand-driven funding mechanism that facilitates and finances skills-upgrading of in-service employees, and provide prospective employees with relevant pre-service and in-service training.
(ii) Organizational restructuring and staff development; and
(iii) Reorientation of the training delivery model to reflect employer defined competencies.
4. Project Management. Implementation and Information Technology (Appraisal: US$ 0.71 million. Revised: US$ 0.55 million. Actual: US$ 0.65 million). This component was to finance support for project implementation arrangements and the development of the IT system and hardware.
During the restructuring in 2013, project activities that were no longer relevant were eliminated:
- Cancelled support and capacity building activities for the E-TVET Council and its Secretariat, as well as support for performance-based planning and realignment of budget allocations.
- Cancelled activities that were premature, including employment demand projects, development of a medium-term expenditure framework, and support to the National Qualification Framework.
- Added new technical assistance activity to support implementation of the National Employment Strategy.
- Technical assistance was added to develop an M&E system to track vocational training interventions.
- Component was dropped due to lack of clarity about immediate future of the Fund, its mandate and sources of financing.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
- The actual project cost was US$ 4.66 million, compared to the original appraised amount of US$ 8.33 million including front-end fee for IBRD. Revised project cost following the restructuring was US$ 7.59 million. Actual costs were lower because of delays in implementation and cancellation of activities during project restructuring.
- The project was financed by a US$ 7.50 million IBRD loan, of which US$ 3.37 million (45%) actually disbursed. During the March 2013 restructuring, US$ 1.21 million of the loan was cancelled, and the revised loan amount became US$ 6.29 million. The amount undisbursed at project closure was US$ 2.92 million.
- The actual borrower contribution was US$ 0.96 million, which is 72% of the appraised amount of US$ 1.33 million.
- March 2013: Level-1 restructuring, six months before closure, to ensure the project's relevance to Jordan's National Employment Strategy which was formally launched in June 2012. Disbursement at this point was US$ 1.86 million (55% of the actual loan amount). The following changes were introduced:
- IBRD loan amount reduced and US$ 1.21 million cancelled.
- Project development objectives and components were revised (see above) and the M&E design was changed.
- Funds were reallocated across disbursement categories.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:
Original objective: High
Revised objective: High
Both the original objective of enhanced internal and external efficiency of the E-TVET sector through employer participation and the revised objective of a realigned E-TVET sector with the National Employment Strategy are in line with the government's National Agenda (2006-2015), which was in 2012 replaced by the National Employment Strategy. The Agenda proposes actions to improve employment, the availability of entry-level workers with the skills needed in the labor market, and training programs to upgrade skills of the employed. The objectives are relevant considering the problems identified in the National Agenda, including that the vocational training sector had produced poorly trained and uncommitted workers, a lack of private sector collaboration, fragmentation of training programs, and the need for a holistic approach. The National Employment Strategy aims to increase labor participation and create more and better jobs. The objectives are also relevant for the government's Executive Development Program which highlights a knowledge-based economy and harmonized education outputs with labor market requirements.
The objectives are also relevant in the context of the Bank's Country Partnership Strategy (2012-2015), which aims to support job creation through developing skilled human resources in science and technology in line with private sector demand. The objectives address key challenges in education such as low enrollment in vocational education (around 13%) and the need to realign vocational education to the demands of the labor market. The objectives are also relevant in a context of high youth unemployment rates and an unmet demand by employers for individuals with vocational skills.
b. Relevance of Design:
Original design: Modest
Revised design: Substantial
The results framework in the original design included relevant activities to strengthen and coordinate technical and vocational education and training reform, and it supported employer-driven programs to attract more youth and ensure their transition into the labor market. The design focused on reforming governance, program development, and financing of the Vocational Training Corporation (VTC) serving students at risk of exclusion. These activities are logically linked to the objective of a more efficient E-TVET sector. However, the design had weaknesses. The original project objective was too ambitious given the project's time frame. The third component received only symbolic support from the project which made it less focused; this component was cancelled during the restructuring. Also, the design did not adequately plan for the financing of E-TVET, which raises concerns about sustainability. Finally, the design proved to be too complex in a context of fading government ownership for this project, and had to be revised.
The revised design had a less ambitious objective of an aligned E-TVET sector with the National Employment Strategy, which is now hosted by the King Abdullah Fund. It also included fewer activities, thereby increasing project focus. The design is relevant for the work of other organizations, including the International Labor Organization and the European Union. Project components are logically linked to the objective. Still, the revised design was too ambitious given the limited project time left and lack of government ownership.
|4. Achievement of Objectives (Efficacy) :|
The project aimed at setting up a new governance framework and a new set of strategic incentives to change the way TVET programs are conceived, designed, set up, delivered and evaluated (ICR, p. 17).
Original objective and design: To enhance the internal and external efficiency of the E-TVET sector by making it more flexible and demand driven through the development of employer community participation.
Modest. There is no evidence that the E-TVET sector became more efficient, including changes to the rate of return of vocational education and changes to its contribution to the labor market and economy. The project has helped develop relevant tools in collaboration with employers, including an employer-driven Occupational Profile Manual which has been adopted by the Center of Accreditation and Quality Assurance, a E-TVET sector performance assessment system, a TVET communication strategy, an employer-driven modality to identify priority sectors for new training programs, and a baseline survey on public perception of the TVET. However, so far these tools have not been used to produce a governance framework and policy directives that could affect sector efficiency.
Revised objective and design: To realign the E-TVET sector with the National Employment Strategy by enhancing the enabling conditions for employer participation in: (i) TVET institutional development; and (ii) skills development program design and delivery by the Vocational Training Corporation.
Modest. There is insufficient evidence of project contribution to enhanced enabling conditions for employer participation. The project helped in the preparation of background work for the three pilot training programs and some enabling conditions were introduced in the Vocational Training Corporation. Enrollment increased but cannot be attributed exclusively to the project. The E-TVET council has not been effective in providing employer-driven policy directives to its stakeholders. Cabinet approval is needed of the new by-laws for the Vocational Training Corporation to allow the introduction of the revised organizational structure.
Outputs relevant for original and revised objectives
- There were preliminary steps taken toward piloting of employer driven business, training, and governance models in three Vocational Training Corporation (VTC) centers, but no actual piloting was conducted. So far, the organizational structure, audit report and staffing plans were accepted in early 2013; but the VTC by-laws are still awaiting Cabinet approval.
- The completion rate of VTC graduates was 73% in 2013, surpassing the 70% target; however, project attribution is not possible.
New output indicators added during restructuring:
- Development and introduction of a TVET Sector Performance Assessment System (PAS) was not met and related reports have not been provided. So far, PAS and data collection tools were developed and approved in early 2013, and a second assessment report was received in September 2013.
- There is no information on increased public awareness for E-TVET; Terms of Reference have been drafted but the survey has not yet been carried out to collect relevant information.
- The enrollment rate in VTC increased by 12% from 2012 to 2013; however, this increase cannot be attributed to the project as most activities under component 2 were delayed to 2013 or have not yet been implemented due to political challenges (see Section 9a).
- The female enrollment rate in VTC increased by 36% from 2012 to 2013 for regular courses and decreased by 42% for short-term courses over the same time period. Project attribution is not possible as component 2 activities were delayed to 2013 or have not yet been implemented.
The following output indicators were not measured and were dropped during the 2013 restructuring:
- Enhanced understanding of occupational opportunities.
- Employer driven occupational profiles developed.
- Restructured VTC centers establish performance indicators.
- Financial resources allocated against performance.
- Policy objectives define feedback from TVET stakeholders.
- Autonomous governance model in place.
- Satisfaction among employers with training delivered through VTC.
- Increased % of non-leavers from 10th grade in total of enrollment.
Outcomes relevant for original and revised objectives:
New outcome indicator added during restructuring:
- Number of employer-driven occupational profiles developed and endorsed by the E-TVET Council increased from 0 in 2009 to 6 in 2011, meeting the target of 6.
- Organizational audit, staffing plan, and organizational structure are planned to be introduced in VTC once the by-laws are approved. So far, organizational structure, audits and staffing plans were accepted in early 2013, and VTC by-laws were sent to Cabinet for endorsement in July 2013.
- The number of employer-driven skills development programs designed increased from 0 in 2009 to 3 in 2013. However, these programs were not delivered and implemented, and therefore the target was not met.
The following outcome indicators were not measured and/or were dropped during the 2013 restructuring:
- Functional E-TVET Council. (An E-TVET Council law was passed in 2009.)
- Improved employer satisfaction of ETVET graduates entering employment.
- Improved VTC internal efficiency through increased trainee/instructor ratio increased from 6.4 in 2006 to 12 in 2013.
- Number of firms receiving training through TEF (10 in 2009).
The economic analysis at appraisal anticipated that the project would contribute to higher employment among TVET graduates due to higher relevance of their skills. This would lead to higher lifetime earnings. In addition, it was expected that unit cost per graduate would decline as a result of the reorganized TVET system, and completion rates would improve. Based on these assumptions, and using a discount rate of 7 percent, the Project Appraisal Document estimates a benefit/cost ratio between 2.0 and 4.0. Rates of return are not reported in the ICR. There is no evidence that these anticipated benefits are materializing.
The ICR (p. 18) identifies some possible efficiency gains from project benefits, including the elimination of supply-driven courses at the VTC which may reduce cost. The ICR estimates possible benefits of about US$ 4.3 million if the reform tools supported by the project are eventually implemented.
Project funds were cancelled during the restructuring, and the project closed on time, which contributed to the efficient use of project funds. However, there were substantial delays in implementation, and lengthy government and Bank procedures substantially delayed the processing of project restructuring, which negatively affected the efficient use of project funds.
Efficiency is rated Modest.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated