Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Secondary Education Project


  
1. Project Data:   
ICR Review Date Posted:
11/06/2013   
Country:
Turkey
PROJ ID:
P066149
Appraisal
Actual
Project Name:
Secondary Education Project
Project Costs(US $M)
 $104.0 million  $104.6 million
L/C Number:
L4767
Loan/Credit (US $M)
 $104.0 million  $104.6 million
Sector Board:
Education
Cofinancing (US $M)
 n/a  n/a
Cofinanciers:
n/a
Board Approval Date
  03/15/2005
 
 
Closing Date
05/31/2010 04/30/2012
Sector(s):
Secondary education (50%), Vocational training (33%), Central government administration (17%)
Theme(s):
Education for the knowledge economy (67% - P) Education for all (33% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Erik A. Bloom
Robert Mark Lacey Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:

The PDO according to both the PAD (p. 5) and Legal Agreement is "to improve the quality, economic relevance, and equity of secondary education to support lifelong learning."

This was modified through a Board-level restructuring on April 7, 2008: "to support the Government's reform of general secondary and vocational education by improving conditions for student learning, establishing a career guidance system, and enhancing foreign language teaching."

This was modified a second time on March 24, 2010 "to support improvements of conditions for student learning in secondary education and in schools in sub-provinces with low enrollment rates."

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 02/09/2008

c. Components:

The number and focus of the components changed throughout the implementation period. The evolution of the components and their costs are shown in the following table. All costs are presented in euros as was done (for the most part) in the project documents.
Cost per Component (in millions of euros)
ComponentAppraisalFirst level 1 restructuring Second level 1 restructuringFinal Amount
Component 1 (Secondary School Reform)64.745.147.035.5
Component 2 (Quality Assessment)5.5 3.52.43.8
Component 3 (Staff Capacity Development)Component not included at approval27.0Component dropped at second restructuring
Component 3 Revised (School Grants)Component not included at approvalComponent not included at first restructuring27.034.6
Component 4 Project ManagementComponent not included at approval4.43.63.1
Contingencies 9.30.00.00.0
Fees0.80.80.80.8
Total (sums may not match due to rounding)80.080.080.077.0

There were several key activities that were supported at various times through out the life of the project. These include:

  • Curriculum reform. Revision of curriculum and textbooks.
  • Provision of information and communications technology.
  • Career guidance and counselling. Providing youth with access to information and counselling services.
  • Student assessment. Development of a system system to collect and disseminate information on student learning.
  • School development. Support to develop management teams as well as provide grants to schools in poorer sub-provinces (after 2008).
  • Training for foreign language. Provision of training in foreign language to teachers (from 2008 to 2010).
  • Vocational training initiatives. Support for development of new initiatives for technical and vocational education (after 2008)

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs
As the above table indicates, total project cost at closure was €77.0 million, €3 million less than the appraisal estimate of €80 million. As well as the two level I restructurings, there were two level II restructurings (July and December, 2011), which primarily transferred resources from component 1 to component 3.
Financing
The approved IBRD Loan was in the amount of €80 million (US$104 million equivalent at the time of approval). The actual amount disbursed was €77.0 million (US$104.61 million at closure). The undisbursed €3.0 million was canceled at closure. There were no other external sources of financing.
Borrower Contribution
The Borrower made no recorded financial contribution to the project.
Dates
The project’s closing date was extended three times as part of the March 2008 and April 2010 level I restructurings, and of the December 2011 level II restructuring. The first, a seven-month extension from May 31, 2010 to December 31, 2010, was to allow the implementation of a new component (Staff Capacity Development), and the expansion of the already existing Project Management, Monitoring & Evaluation component. The second extension was for 12 months, from December 31, 2010 to December 31, 2011, to allow time for the implementation of the newly added School Development Program. The third and final extension, from December 31, 2011 to April 30, 2012 was in order to complete all activities under the SDP and under the Secondary Education Reform component, in particular the delivery of goods and equipment needed for training associated with curriculum reform. The project closed on April 30 2012, 23 months behind the original schedule.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Original objective (03/15/2005 to 02/09/2008). Substantial. At the time of approval, the objective was well aligned with the government's and Bank's priorities. The 8th National Development Strategy (2001 to 2005) prioritized actions that contributed to the knowledge economy. The government's 9th National Development Strategy (2007-2013) emphasized the importance of skills-based secondary education. By project closure, in April, 2012, the Bank had excluded education (with the exception of early childhood development) from its country strategy for Turkey. In large part, this reflected the perceived issues associated with the project under review. The original objectives are consistent with the 9th Development Strategy and remain relevant for the government.

First revised objective (02/09/2008 to 03/24/2010). Substantial. The first revision to the objective emphasized the importance of reforming secondary and improving conditions for student learning. This remained consistent with the 9th Development Strategy, which identified improving quality as one its priorities in the area of human development. It specifically mentioned the importance of improving curricula and reforming the education system.

Second revised objective (03/24/2010 to 04/30/2012). Substantial. The second revision of the objective is similar to the first in that it emphasizes the conditions for student learning. It remains relevant to the government's strategy under the 9th Development Strategy, which makes specific reference to improving curricula and reforming the education system.

b. Relevance of Design:

Original objective (03/15/2005 to 02/09/2008). Negligible. The original objective was quite ambitious in its scope, essentially covering all aspects of quality at the secondary level. As designed, the project did not address these larger issues and focused on a limited set of activities. These activities were not well articulated and it is not clear how they would have led to the desired outcomes. While the Bank did consider alternatives like a development policy loan (DPL) or a series of projects (APL), these were not adopted. At preparation, the project did not have a strong results framework that brought together the different approaches in the project.

First revised objective (02/09/2008 to 03/24/2010). Modest. The objectives were revised to better reflect the project's size and capacity. While the revised design was more realistic it was still ambitious with its focus on supporting "reform." The project's activities were aligned with the three sub-objectives. The results framework was largely adequate but included some indicators (i.e. test scores) that were unlikely to improve during the project period.

Second revised objective (03/24/2010 to 04/30/2012).Substantial. The main activities planned at the second level I restructuring (primarily redesigning secondary curricula, developing student student assessments, and vocational training initiatives) directly contributed to the revised objective of improving learning conditions in secondary education. Curricula form a central part of the learning process and the project design included activities to develop new curricula, train teachers in applying the updated curricula, and develop and distribute materials to schools. A substantial amount of the project was allocated to school grants. Evidence from other countries (for example, Mexico or Nicaragua) suggest that school grants and autonomy can play a role in improving the conditions for education quality.


4. Achievement of Objectives (Efficacy) :


Original objective (03/15/2005 to 02/09/2008). .."to improve the quality, economic relevance, and equity of secondary education to support lifelong learning." Negligible.
There was very little progress during this period and only 1.8 percent of the Loan was disbursed. There was no evidence of improvements in quality (for example, through student assessments) or that there was greater equity. Given the limited activity, it would be in any event have been difficult to attribute any such changes to the project.

First revised objective (02/09/2008 to 03/24/2010). "...to support the Government's reform of general secondary and vocational education. by (i) improving conditions for student learning, (ii) establishing a career guidance system, and (iii) enhancing foreign language teaching" Each of the three conditions are treated as separate sub-objectives since they reflect outcomes rather than simply activities or outputs.

For improving conditions for student learning, the project is rated substantial.

Outputs
  • The project supported the development of new curricula for 49 (out of 66) secondary subjects. Of these, 14 were implemented in the classroom by March 2010, with textbooks being published (page 19 of the ICR).
  • The project initiated the Vocation Training Initiative that aimed to develop new options for technical education and school-to-work initiatives. According to the ICR (page 20), there were 433 applications received, of which 46 were accepted.

Outcomes
  • As described below, the new curricula did have a positive impact on improving the conditions for student learning. Investment under this phase contributed to the development of the new curriculum described below.
For establishing a career guidance system, the project is rated substantial.

Outputs
  • The project supported the creation of a Career Information System to provide secondary students with information about employment to help them to plan their education better.

Outcomes
  • The Career Information System has remained as a central part of the government's reform strategy for secondary education. It has exceed expectations in term of its usage.
For enhancing foreign language teaching, the project is rated negligible.

Outputs and outcomes
  • Improved training for foreign language teachers was meant to be a major initiative of the project under this restructuring. However, this activity was not carried out due to disagreements between the Ministry and the Bank on the composition of the shortlist and the project had no impact on the quality of foreign language teachers in Turkey.

Second revised objective (03/24/2010 to 04/30/2012). "to support improvements of conditions for student learning in secondary education and in schools in sub-provinces with low enrollment rates." This calls for the division of the rating into two questions: (1) Did the project support the improvement of conditions for students nationwide? and (2) Did the project support the improvement of conditions for students specifically in schools in sub-provinces with low enrollment rates?

For support the improvement of conditions for students nationwide, the project is rated substantial.

Outputs
  • The project supported the development of new curricula for the remaining 17 subjects in secondary education (leading to full coverage of all 66 areas) as well as 1,151 vocational education modules. Of these, 62 secondary curricula and 924 vocational education were fully implemented. This meets the targets for curricula developed (66 out of 66 curricula) and exceeds the target for vocational education (1,151 modules compared to a target of 1,000). As noted above, many of the outputs were developed under the previous objective period although most of the implementation was carried out under this objective.
  • As mentioned above, the project also supported the development of a Career Implementation System which involved cooperation between the Ministry of National Education and the Turkish Employment Agency. By the time the project closed, 1.3 million users had accessed the system compared to the target of 300,000. The project also supported vocational training initiative grants to 46 proposals.
  • As planned, the project supported the dissemination of assessment results (through piloting the Student Achievement Assessment Exam) as well supporting eight policy studies.
  • The project supported school autonomy, through the establishment School Development Management Teams in all secondary schools and trained 15,000 educational staff in school plans.

Outcomes
  • The development of new curricula for both general and vocational education can contribute to improving the conditions for student learning if these curricula of high quality and are implemented in the classroom. Most of these curricula were being implemented by project closing (with teachers trained and textbooks in students' hands). In order to evaluate the curricula, the project commissioned an independent review (Ministry of National Education, 2012. A Research into Assessment of the Beneficiaries of the Curriculum Reform of the Secondary Education Project), The project team provided IEG with a copy of the review. The review is based on research and analysis by recognized experts (led by the University of Birmingham (UK) with four other Turkish and international universities) who conducted interviews with teachers, employers, and students and carried out specific case studies. It considered five aspects of the of the curriculum: context, competency framework, content, learning-teaching process, and measuring process.
  • The review finds the new curricula to be an advance over the previous curricula. In general, the previous set of curricula was insufficient for the needs of secondary education while the updated curricula was at least partially sufficient in terms of its theoretical framework and competency framework and wholly satisfactory in terms of its content. The review also found that the curricula were sufficient to improve learning-teaching process and as a basis on which assess the progress of students.
  • Introducing student assessments is generally seen as an important contribution to an improving student learning environment. The development and dissemination of student results play a major part in secondary education reforms in countries as diverse as the United States, Mexico, Kyrgyzstan, and Rwanda. In the case of Turkey, the introduction of student assessments along with the new standards that were part of the revised curricula can be seen as part of a secondary school reform.

For support the improvement of conditions for students in sub-provinces with low enrollment rates, the project is rated modest.

Outputs
  • Established a School Development Program that provided school grants in schools and sub-provinces with low enrollment rates. An independent program assessment (School Grants in Turkey: An Assessment of the School Development Program, 2012, provided by the Project Team and reported in ICR) shows that program was targeted as planned.
  • Through the School Development Program, the project provided school grants to 3,671 schools (compared to a target of 2,710 schools). The program assessment showed that around 40 percent of this was spent on education equipment and books, around 40 percent was used sanitary facilities, small scale renovation, and related equipment (for example, heaters), and around 20 percent was used for academic initiatives (for example, training for teachers, tutoring for students, counseling services, and field trips).
  • The project also purchased and distributed ICT equipment to 1,629 schools. This largely consisted of projectors and photocopiers. This represents a change from the initial plan to equip 322 ICT classrooms as enunciated in the March, 2010 Project Paper.

Outcomes
  • It is difficult to measure the project’s impact on schools with low enrollment, which would be primarily through the School Development Program. The program assessment carried out a random sample of schools and a case study evaluation. The case study evaluation showed that a wide group of beneficiaries (head teachers, teachers, and parents) were satisfied with the improvement in the quality of the school. Likewise, it showed that head teachers of poorer performing schools were “empowered” to developing planning for dealing with low school quality and the high dropout rate.
  • The overall improvement in the curricula (described above) benefits students in poorly performing schools. The new curricula focus on technical competencies. The improvement in teaching quality implied in the curricula reform should also make poorer schools more attractive.

5. Efficiency:


The overall efficiency is rated modest. This reflects slow disbursement until 2010 followed by a period of faster disbursement in line project design.

The ICR contains a detailed economic analysis (annex 3) which argues that the main inefficiency that the project faced was the delay in implementation. The ICR also argues that there was little attempt to create synergy across the various activities. Due to the changes in the project and limits with the existing data, the ICR argues that it is difficult to carry out a detailed cost-benefit analysis.

Under the original objective, the loan tied up resources that could have been used more productively. The low level of disbursement and the limited progress indicate a low level of efficiency. In addition, the project was costly to the government of Turkey and the World Bank, both of which were obliged to invest significant resources to supervise the project. On a positive note, loan resources were not wasted and were available after the project was restructured.

Under the first and second revisions of the objectives, the funds were disbursed. The disbursement was particularly fast under the second revision and in line with the Under the second revision of the objective, the project design contained features that would be expected to enhance the efficiency with which Bank resources were used. However, there is no evidence that this in fact occurred.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


As a result of two major restructuring, there were three different project development objectives. Following standard procedures, this requires weighing the relevance of each objective using the percentage disbursed. The follow table indicates the weights used for the evaluation (similar information is presented in Table 3 of the ICR).

Original objective (03/15/2005 to 02/09/2008). Under the original objective, there was no real activity and thus no progress towards the objective ambitious. With a substantial relevance of objective and a negligible rating for the relevance of design, as well as a negligible rating for efficacy and a modest rating for efficiency, the outcome for the original objective is rated as highly unsatisfactory.

First revised objective (02/09/2008 to 03/24/2010). Project performance improved under the first revised objective, with a substantial relevance of objective, and a modest rating for the relevance of design, with two substantial ratings and one negligible rating for efficacy, and a modest rating for efficiency. The revised objective is rated as moderately unsatisfactory.

Second revised objective (03/24/2010 to 04/30/2012). Progress improved after the second revision of the objective. With a substantial relevance of objectives and of design, with one substantial rating and one modest rating for efficacy and a modest rating for efficiency, the second revision is rated as moderately unsatisfactory.

DateAmountPercentage WeightRating during period
03/15/2005 to 02/09/2008$1.9 million1.8%Highly unsatisfactory
02/09/2008 to 03/24/2010$19.1 million18.3%Moderately unsatisfactory
03/24/2010 to 04/30/2012$83.6 million79.9%Moderately unsatisfactory
03/15/2005 to 04/30/2012$104.6 million100%Moderately unsatisfactory


a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

Overall, there appears to be a high level of national ownership of the activities supported by the project under the second revision of the objective. The initiatives supported by the project have been well received by the secondary education system and are likely to remain in use for the foreseeable future.

The Ministry of National Education plans to continue supporting curriculum development for both general and vocational secondary education through training, textbooks, and the development of learning materials (Annex 7 of the ICR).

The Career Information System, which provides information to secondary school students and graduates, has proven to be quite popular and the number of users exceeding expectations (p. 19 of the ICR). The Ministry indicated that the system is still active and continues to be utilized after the project closed (Annex 7 of the ICR).

The Beneficiary Survey on the Vocational Training Initiatives indicated that most of the beneficiaries expect to continue using the programs that were developed with the project's support (Survey Report provided by the project team).

Several other elements seem less secure. While the School Development Program was fully implemented as designed, it has not been renewed (p. 15 of the ICR). While the government has continued its commitment to decentralization and school based management, it has not continued its effort to mainstream this activity (p. 27 of the ICR).

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

Preparation was a long process that lacked strategic focus. It took four years from the time of the concept note to the project’s approval and an additional year until effectiveness. In a dynamic, middle income country such as Turkey, it is unlikely that the ideas expressed in the concept paper would remain wholly relevant after this time. According to the ICR (page 2), the focus of the preparation appeared to be driven by largely by a desire to maintain a foothold in the education sector rather than towards any specific strategic goal.

The preparation supported significant analytical work but this did not appear to be utilized in the project design. Little effort was spent on supporting strategic planning or identifying key areas of the government program for financing.

The discussion of risks in the PAD did not include the fiduciary risks, and focused more on leadership and intra-agency communication (page 11 of the PAD and Annex 6 of the PAD). The PAD praises the financial management arrangements as highly satisfactory (page 13 of the PAD). Likewise, there was no discussion of (at the time of preparation) the lack of Ministry ownership, despite signs such as lack of interest in a preparation grant (page 7 of the PAD) or the lack of detail in the Ministry's project document (page 11 of the PAD).

The discussion of lessons learned was superficial and did not include any specific lessons from any of the previous Bank projects in the sector. Given that many of recent education projects in Turkey had low ratings, this is a shortcoming in the preparation.

While the ICR indicated that there was cordial dialogue during preparation, the Bank and the government did not agree on the immediate inclusion of a component supporting foreign language training for teachers, which appeared to be a major priority of the government. The Bank indicated that this activity could be added at a later stage. However, once the Bank had agreed to its inclusion, it is not clear why it was not added during preparation.

The M&E framework was unrelated to the activities. While the planned activities were likely to have a positive impact, it is hard to see the relation between the activities and increasing quality at a national level.

Quality-at-Entry Rating: Highly Unsatisfactory

b. Quality of supervision:

According to the ICR (page 28), the World Bank team largely focused on addressing design shortcomings during the first five years. Most of the dialogue focused on dealing with short-term design issues with little focus on the project's development or strategic issue. As a result, the team often endorsed actions that were not technically well grounded or practical given the different procurement guidelines used by Turkey and the World Bank. This was the case, tor example, with foreign language training.

The situation improved considerably after the second restructuring (March, 2010). The restructuring itself adjusted the project's objective to a more achievable level. Following this, the Bank worked closely with the government to implement the project. The Bank showed necessary flexibility and helped facilitate several complicated procurements. According to the project team, the Bank responded to the earthquake in October 2011 by encouraging the School Development Fund to focus on affected areas, consistent with the project's objectives. While the project team leader was based at World Bank headquarters, the Bank largely decentralized the supervision team to the Turkey Country Office. This facilitated contact with the Borrower (page 28 of the ICR).

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The government approved the project despite the lack of interest shown by the Ministry during preparation. This led to problems during implementation and contributed to a 15 month delay in effectiveness.

While there was no requirement for the government to provide financing, it was supportive in approving the various project restructurings. The Ministry of National Education was able to work successfully with other Ministries. On the policy level, several vocational programs were transferred to the Ministry from other agencies as part of the effort to strengthen vocational education (annex 7 of the ICR). The Ministry also worked with the Turkish Employment Agency to implement the Career Information System.

The ICR states that the governments "inability to resolve issues in a timely fashion" resulted in limited implementation during the first five years of implementation.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:

The implementing agency was the Ministry of National Education, particularly its Project Coordination Center (PCC). According to the ICR (page 29), the weak performance of the Ministry during preparation and the early stages of implementation reflected the small degree of ownership it felt for the project. The fact that the project offered no additional resources and did not support the Ministry’s priority (training for foreign language teachers) contributed to this.

After the government and the Bank agreed on the first major restructuring in March 2008, which included foreign language training, the Ministry started to take advantage of the project. Although the foreign language training component ultimately failed due to differences between the government and the Bank over procurement, the Ministry displayed greater ownership, and the disbursement rate increased.

After the second revision of objectives, in April 2010, the Ministry showed still more ownership of the project as it had become involved to a greater extent in the design of the project activities. It strengthened the PCC, and financial management improved significantly. The Ministry was able to take advantage of the project’s resources and technical assistance to implement different activities as shown in the borrower’s comments. As a result, the Ministry finalized the preparation of new secondary curricula and vocational education modules. This also included preparing supporting textbooks, learning materials, and training for teachers. The Ministry was able to set up a School Development Fund to support the school-based management initiative as well as introducing a career guidance system (see Annex 7 of the ICR).

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The initial M&E design was focused on high level indicators at the outcome level, including improving performance on student assessments and reducing variance in student assessments (as a measure of increased equity).Output indicators included an increase in employment participation of beneficiaries of guidance and counselling. While these indicators are measurable, the extent to which the project can contribute to the goals measured by the indicators was unclear. In this sense, the indicators did not provide much information to the project or guidance.

After the first major restructuring, additional indicators were added. These were vague and difficult to measure in practice (for example, improved foreign language capacity). At the same time, the intermediate indicators were reduced in scope and in many cases reflected lower level actions, more akin to inputs or activities than results (for example, in-service training on new curricula provided to all teachers or equipment provided to schools).

The second major restructuring developed a set of indicators that were more closely aligned to the new objectives. At the same time, many of the indicators were still difficult to measure objectively at the outcome level (for example, secondary education curricula more aligned to student competencies). The intermediate indicators were generally well-prepared and at the right level (for example, student achievement results are reported and shared with stakeholders and public). Data were to be collected by the Ministry's existing information system and by the Project Coordination Center, which was also responsible for preparing the results.

b. M&E Implementation:

To the extent possible, the project monitored indicators that were generated by external sources (i.e. student assessments) or were administrative data from the project (for example, the distribution of the ICT equipment to schools). The project was less successful in carrying out independent evaluations and surveys as proposed in the results framework, including the independent assessment on the curricula. After the second major restructuring, more of these evaluations were carried out, including a beneficiary assessment of the School Development Program and of the new curriculum. The project also financed A Research into Assessment of the Beneficiaries of the Curriculum Reform of the Secondary Education Project, the results of which became available after closure.

a. M&E Utilization:

There is little evidence that the data collected from M&E was used in the project. Some of the evaluations carried out at the end of the project were used by the Ministry in understanding the project's impacts and its efforts to sustain some of the activities supported by the project.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

This was a Category "C" project for purposes of Envrionmental Assessment. No safeguards policies were triggered.

b. Fiduciary Compliance:

According to the ICR (page 14), the project's audit reports were submitted in a timely fashion and were unqualified. At the same time, the internal financial management capacity of the Project Coordination Center was quite variable and led to frequent concerns on the part of the Bank team.
The ICR reports that procurement was carried out according to Bank guidelines. However several major procurement packages were cancelled due to lack of agreement with the Bank on both technical and fiduciary grounds. There were no reported cases of misprocurement.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Unsatisfactory
Moderately Unsatisfactory
With around 80 percent of the disbursement occurring after the second revision of the objectives, the rating during this phase drive the overall outcome rating. Evidence of improved performance during the last phase is taken mainly from an independent review, which was not reported in the ICR. 
Risk to Development Outcome:
Significant
Moderate
The Ministry plans to continue its support for activities supported by the project, particularly after the second revision of objective.  
Bank Performance:
Unsatisfactory
Unsatisfactory
 
Borrower Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

The ICR presents an extensive series of relevant lessons that were derived from project preparation and implementation (pages 30-32). These lessons are drawn from the project's implementation experience, as well as reflecting specific technical issues related to the School Development Program and vocational training. Three key lessons from the implementation of the project are:
  • Ownership of the implementing agency remains the single most important factor in determining the success of a project. Clearance and support from central Government agencies are a necessary but not sufficient condition for success.
  • A significantly flawed initial design is difficult and costly to correct during the course of implementation.
  • From a government perspective, establishing a relevant role for the Bank in a sophisticated country needs careful discussion of where the Bank’s value-added is and how it can be maximized.

14. Assessment Recommended?

Yes
Why?
The project was designed as the centerpiece of the Bank's engagement with Turkey and it clearly had many shortcomings. A further evaluation could identify what areas of the project worked.

15. Comments on Quality of ICR:


The ICR presents a sound analysis of the project. This includes a comprehensive description of the implementation of the project, from preparation to closing, explaining the initial constraints that the projects faced. The ICR also as well as the various directions that project moved during implementation. This includes a good description of the various sub-components and activities.

The ICR is candid in describing the strengths and shortcomings during preparation and implementation. In particular, the description of quality of entry is quite through.

The ICR shows a clear understanding of the difference between meeting targets as opposed to the attribution to the project for the outcomes. The ICR has a clear analysis of the impact of the project in a variety of areas. Each of the ratings is well justified and explained. The lessons are well-organized, including both general issues and technical issues, and provide reasonable to future operations.

The ICR also has a comprehensive economic analysis that provides an excellent description of the economic impact of the project. The ICR can be used as a model for future projects regardless of the rating.

The ICR's assessment of outcome is weakened by its lack of access to an important end-of-project. This review, which was carried out with project support, provides important data on the impact of many of the activities supported by the project. In addition, the ICR's estimation of the final rating for outcome does not seem to reflect the guidelines regarding methodology for calculating ratings when the objectives change.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions